Serverless support

[steebchen]

Problem

Hatchet currently doesn't work on serverless infrastructure (such as lambda, Vercel functions and so on), as the worker needs to be a long-running process and thus only works on VPS or actively running containers.

Solution

We could add support for serverless by adding a concept of serverless workers and allowing to define an HTTP URL, which will be executed for each run.

The user would add an API route using a simple hatchet API to handle the incoming jobs, and then define their job as usual.

Thoughts

• Does each step invoke a new HTTP request? Assuming yes.
• The liveness of the serverless worker could be determined if there are more than X failed requests in the last X seconds/minutes
• Initially only TS SDK is supported as that's the most common serverless language

Caveats

• Most likely this is best for jobs which take less than 30s; serverless is often best for short lived workflows. If you have big batch jobs, splitting them into multiple steps could help.
• We need some kind of middleware/wrapper/interceptor, one for serverless and most likely one for each serverless platform due to vendor differences

Spec (WIP, edited)

Abstractions

• Webhook Controller
  • this uses the same concept of a regular worker, except
    1. the user defines a webhook URL when creating the workflow
    2. no actual business logic / handling of the job is implemented with the SDK interaction
• Serverless adapter: directly compatible with webhook controller, just helpers for DX
• Serverless runtime adapters (cf workers, vercel, etc.)
  • also just helpers for DX
  • needs good docs – as for example, vercel hobby plan serverless execution is limited to 10s in non-edge functions

[grutt]

Thanks @steebchen! Yes I've been thinking about this too.

To address your throughts:

Does each step invoke a new HTTP request? Assuming yes.

• I think yes, otherwise serverless timeouts would be a major problem

The liveness of the serverless worker could be determined if there are more than X failed requests in the last X seconds/minutes

• this makes sense

Initially only TS SDK is supported as that's the most common serverless language

• I've heard a number of FastAPI serverless usecases, I agree we should start with TS but quickly support others

Caveats

• Most likely this is best for jobs which take less than 30s; serverless is often best for short lived workflows. If you have big batch jobs, splitting them into multiple steps could help.
  One additional thought here is durable resume which should work well with child workflows. We may need to put a bit of thought into how we're handling retries since timeouts and exceptions are handled the same right now.

• We need some kind of middleware/wrapper/interceptor, one for serverless and most likely one for each serverless platform due to vendor differences

Yes, I think we break this up into two steps:

1. expose the action handler so it is not as tightly coupled with action listener
2. write connectors for common serverless frameworks. I think we start with next but open to thoughts here. Effectively we'd just write a POST handler that handles all requests on a specific path.

We'd also need to think about registering this route with Hatchet and auth (probably signing and verifying the request).

[grutt]

One additional thought after posting... I think we zoom out from serverless here and think about this feature as a webhook controller. I've heard a few times folks want to start using hatchet without needing additional infra and this route would work equally well as a route on a long lived api.

[abelanger5]

@steebchen I like this proposal and would love to prioritize this. It does seem rather simple to add a webhook controller after assigning a step run to a serverless worker, but I think the biggest problem is registration:

1. Have you given any thought to how the serverless worker gets registered in the first place? Does the user specify the HTTP URL and a set of workflows (or more specifically, actions) that this worker should execute? What happens if the user wants to change the actions that a worker/endpoint can execute?
2. Related to above, how do the workflows get registered in the first place? Since right now, they get registered when the worker is booting up.

There are a lot of similarities to running a database migration, so perhaps we provide a function which registers workflows and a serverless worker that gets run in the same place as a DB migration would? Seems like a lot of people do this as part of CI/CD in the serverless world, which seems not pleasant, or you expose an API handler for it...

[steebchen]

Regarding this 'webhook controller', I think one open question is if this controller should behave like a worker currently (registering workflows) and only when used via the serverless wrapper of this webhook controller where registration has to be defined like db migrations.

[steebchen]

I'm actually wondering about the specific use cases of a Webhook Controller which is not serverless, aka which acts as a regular worker, as I can't really think of any right now.

What's the selling point to use it? I currently only see benefits for either a normal hatchet worker as we have today, or a serverless one which works just with URLs and no extra long-running config.

Can you elaborate @abelanger5?

[abelanger5]

Good point, I believe so as I don't see any other way. So the user would have to run the workflow definition code at least once and ideally in CI or whenever they update that definition, which includes the workflow details as well as the webhook URL. It assumes the webhook is live, and marks it as offline if there are more than X failures (TBD).

Yep, that sounds right 👍

What's the selling point to use it? I currently only see benefits for either a normal hatchet worker as we have today, or a serverless one which works just with URLs and no extra long-running config.

This was @grutt's suggestion, so will let him respond with thoughts!

I don't have a strong opinion on this, but I think the primary question is whether we couple the logic which registers this as a "serverless worker" with the logic which sends the webhook. I also don't see a case where we'd send webhooks to start a step run to non-serverless workers. You might want to check out the following for some discussion: https://news.ycombinator.com/item?id=39644543.

Last note - perhaps we call these "HTTP-based workers" - serverless is a little confusing for me.

[grutt]

I have had a number of conversations where folks want to start writing/testing workflows without needing to provision any new infra for a project. Path would look like local --> "endpoint worker" --> long lived worker. I think its the same design as Serverless, but its not guaranteed to be on Serverless infra (i think of lambda or Vercel). This is more of a naming problem than a design problem.

Good point, I believe so as I don't see any other way. So the user would have to run the workflow definition code at least once and ideally in CI or whenever they update that definition, which includes the workflow details as well as the webhook URL. It assumes the webhook is live, and marks it as offline if there are more than X failures (TBD).

Maybe we should expose a health check endpoint (or special health check payload to the registered endpoint) for this? We can hit the endpoint on some interval to register or as part of the CI/CD?

[steebchen]

ok so basically we do not need another abstraction for webhooks/http and serverless, we just call them HTTP-based workers which works for both docker/VPS setups and serverless, and that's it. Those workers would not be long-running (in terms of uptime detection and compatibility with serverless etc.). Is my understanding correct?