Hatchet

How do you ensure that your web app never drops a single user request (particularly when those requests can be very complex, like triggering an AI agent)? It comes down to two simple paradigms: 1. Acknowledgements 2. Separation of concerns Let’s trace the path of a user request which triggers a complicated (perhaps LLM-backed) task: 1. A user clicks a button on your web app, which sends an HTTP request to your server. 2. Your server writes some data to the database for that request, and then creates a new task for that user. 3. Your task runs and updates the database when it’s finished to indicate that the work is done. 4. The user sees the results of the task in your web app. Overall, this seems pretty simple — where can this go wrong? - Your function restarts or crashes when processing the user request - Your function stores the data for the request, but never actually invokes the task - Your task starts, but never completes because of a hardware failure, running out of memory, or a bug in your code - Your task completes, but never gets stored in the database. While some of these scenarios might be unlikely when you just have a handful of users, you’ll start to see these issues more frequently as you scale. How do you solve this? 1. Acknowledgements and retries - every time you successfully process a request or message, you acknowledge it — either to the user in the form of a 200-level response code, or to a broker in the form of an ACK. If you fail to acknowledge a message within a time interval, or you negatively acknowledge a message (i.e. 500-level error), the caller retries the request. Even better if your functions are idempotent and you retry using exponential backoff with jitter. 2. Separation of concerns - different workloads have different runtime requirements. Spawning a very complex, long-running task in the same process as your API handler can cause headaches down the line, in the form of out-of-memory errors, saturating shared resources like file watchers or thread pools, or causing high latency due to intensive tasks eating up your CPU. The simple solution here is to separate your API from your workers, with a task queue (like Hatchet!) that handles ACKs, retries and recovery from failure. This also sheds some light on why using Postgres as a task queue can be so powerful — you remove certain classes of failure entirely when you enqueue a task as part of the same transaction that you’re actually writing user data. — We're building Hatchet - an open-source background worker framework (backed by Postgres) to build reliable AI apps in Python, TypeScript, and Go.

Most early-stage software companies start with a simple architecture: ``` frontend <> api <> db ``` But as your service grows, this pattern starts breaking down... ?? The Problems 1. Request Cancellation: Users closing browsers or navigating away terminate in-progress operations 2. Processing Time Bloat: Complex operations start exceeding reasonable HTTP timeout limits 3. Resource Constraints: API servers can struggle with compute-intensive tasks while handling regular traffic ?? Enter Background Workers Background workers run in a separate processes that handle time-consuming, resource-intensive, or mission-critical tasks asynchronously. Here's how they transform your architecture: ``` frontend <> api <> db ___________↓ ___________worker queue ___________↓ ___________worker pool ``` Why? 1. Reliability ??- Jobs persist even if users disconnect ??- Retry mechanisms handle transient failures (i.e. work can resume on a new worker) ??- Job state tracking enables progress monitoring and improved observability 2. Scalability ??- Offload heavy processing from API servers ??- Independent scaling of worker resources ??- Better resource utilization through job queuing ??- Better technology utilization choosing the right "tool for the job" ?? When should you think about adding background workers? 1. Task Duration > 1-2 seconds 2. High CPU/memory usage tasks 3. Batch processing 4. Critical operations needing retry logic 5. Complex work that has multiple discrete steps --- We're building Hatchet - an open-source async compute platform to build reliable AI apps -- replacing legacy solutions like Celery for Python and Bull for Node. What's your experience with background workers and when is the right time to implement? ??

Why did we choose Python, TypeScript, and Go as our first 3 languages for our Hatchet SDK? Our first language was GoLang, primarily because of its performance profile, strong type safety, and ability to handle concurrency incredibly well. It also didn't hurt that it's what Alexander was most comfortable with. We quickly learned that while Go is a great language, most AI startups are building in Python with FastAPI or TypeScript with Next.js, so naturally we expanded support for both. For Python, we're seeing folks make the shift from ML and data science (where Python rules) into application development. It's often challenging to wrangle Celery while scaling, manage AIO, or designing more complex workflows. For TypeScript, we're seeing teams hit limits with timeouts on edge functions or lack visibility with async tooling like BullMQ. -- ?? The most interesting thing: we're seeing customers move between these languages and mix-and-match as they scale – i.e., adopting Go for higher-throughput ingestion where Python starts to consume more resources or break. Did we make the right call? What SDK should we build next? ?? — We’re building Hatchet - an open-source async compute platform to build reliable AI apps in Python, Typescript, and Go.

?? Product Update ?? This month, our users are on track to process nearly 1 billion tasks on Hatchet Cloud. While most queues and workflow execution platforms are good at displaying either aggregate metrics or individual run history for debugging, most tools aren't optimized for both -- Hatchet is. This week, we’re launching: 1. A new activity overview page which allows you to get a birds-eye view of workflow failures and successes 2. Within each workflow, a full event history containing error traces and timing information, allowing you to debug problematic tasks 3. An OpenTelemetry integration for our Python SDK which automatically sends traces to your OpenTelemetry collector, with prebuilt queries for tracking high latency tasks. You can sign up on Hatchet Cloud to try out our new monitoring features today. -- We’re building Hatchet - an open-source computing service for async and background tasks.

Just 5 years ago web apps were relatively simple – work could be done on the main thread as part of the request, and longer jobs could run as a background task (often overnight). AI is changing this ... We’ve been noticing Hatchet users building AI apps with an architecture that resembles background tasks, but with the user in the loop (often thousands of times per day). This work usually takes the shape of a human task with AI agents reasoning about large amounts of data, instead of just fetching the data and letting humans do the reasoning. If these requests take too long to give the user a sense of progress, they leave. Here’s what we’re seeing as the key problems in these systems: 1?? Software processes are getting more distributed and data hungry by necessity: - RAG agents load in and evaluate 100s of candidate documents in parallel - AI model inference is expensive and time consuming to load models, handle partial results, and timeout/retry on failure - Document generation with real-time progress and preview capabilities - Code generation coordinating multiple model attempts with early stopping - Image processing that streams incremental, low-res images back to users The common thread? They're all workflows that need to do work off the main thread AND provide real-time user feedback. ?? 2?? Sophisticated schedulers like Temporal or Step Functions are often too slow, with 200-500ms scheduling latencies. When you need to coordinate multiple services and get results back to users fast, every millisecond of queue latency compounds. Engineers end up building complex bypass systems mixing queues for reliability with direct API calls for speed. 3?? Current patterns all have tradeoffs for keeping the user in the loop: - Pub/Sub systems: Need to maintain separate Redis/Kafka clusters for streaming, manage connection pools, and write complex error handling for missed messages - WebSockets: Socket management at scale requires sticky sessions or distributed connection tracking, plus fallback mechanisms for reconnects - Event-based processing: Simpler than WebSockets but still needs a separate event source service and handling for backpressure - Long Polling: Extra DB load from constant status checks, eventual consistency delays, and cache invalidation headaches -- We’ve built Hatchet to be fast enough to handle near real-time with built-ins so you can stream state from any running workflow process without additional infrastructure or glue code. Curious to hear your thoughts on this. Have you faced these coordination challenges? What patterns worked for you? ??

?? Announcing Hatchet Lite - a lightweight installation of Hatchet for local development. Get up and running with it here: https://lnkd.in/eH-UJmR7 ? Bundles our UI, API, and engine components in a single image ? Deploys natively on both arm64 and amd64 ? Single-command installation via docker-compose up We'd love your thoughts and feedback!

?? Hatchet is finally open-access! For the past 6 months, Gabe and I have been working hard on our open-source task queue, and working with a few select companies on our hosted version. Hatchet Cloud is now available for anyone to try - including a free tier which lets you run 10k task executions per day! ?? Link in comments - we’d love to hear what you think! --- The backstory → Hatchet started as an idea to build a developer-friendly version of Temporal. This was based on my previous experience of running millions of Temporal workflows/month at?Oneleet (YC S22)?as well as managing task queueing infra on behalf of users as CTO at?Porter. For the initial YC application, we pitched it as a “Workflow management system for developers” (It turns out this is a terrible one-liner, as we quickly learned that “workflow” is one of the most overloaded terms in software. And “Workflow management system” makes it sound like an enterprise tool.) We also built a version of Hatchet over a weekend and posted it on Reddit the next day. Despite the questionable one-liner, we were accepted into the YC W24 batch, and went into the batch trying to sell our product as a?workflow engine?which enables?durable execution. But after chatting with a bunch of technical founders, we learned a few things: 1. “Workflow engine” isn’t something that busy technical founders or startup engineering teams are thinking about.?Most people that we talked to had solved background task orchestration with tools like Celery for Python, BullMQ for Node, or perhaps a home-brewed Postgres task queue. 2. People building on top of LLMs tend to adopt a distributed queue much earlier?than a traditional web app?that primarily reads/writes from a database. LLM apps are much “heavier” from a processing perspective due to slower API calls and a heavy need for ingesting/indexing external sources of information, like documents or codebases. Because of this, many LLM apps have a usability/latency problem with time-to-first-token and incremental result streaming becoming a high priority. 3. Most people don’t need durable execution, at least not early on. 90% of use-cases are solved with a caching layer and idempotency. The tradeoff of needing to work in a deterministic context generally isn’t worth the higher learning curve and non-intuitive programming paradigm. After several iterations of re-positioning — including an attempt at?wrapping Hatchet with an LLM prompt playground?— it started to click when we started talking to users about?the need for a task queue, instead of a workflow engine with durable execution. We started to see adoption, first from other YC companies in the batch, and then?on Hacker News, where we reached number 1 and stayed there for the better part of a day. --- Since our HN post, we've built a ton of features - child workflows, support for global rate limiting, event streaming, and more. Try it out and let us know what you think!