Task queue
A task queue is the backend pattern where work units are submitted to a queue, picked up by workers, and processed asynchronously — Celery, BullMQ, Sidekiq, AWS SQS, Cloud Tasks, Trigger.dev, Inngest are 2026 examples.
Synchronous request → response works for sub-second work; longer tasks (sending emails, processing images, calling slow LLMs, running batch jobs) need async patterns. Task queues solve this: the web request submits a task, returns immediately, workers process in the background. Implementations: Redis-backed (BullMQ, Sidekiq), Postgres-backed (Hatchet, Graphile Worker), cloud-managed (SQS, Cloud Tasks), full-platform (Trigger.dev, Inngest, Defer). Production gotchas: dead-letter handling for permanently failing tasks, retry strategy (exponential backoff vs immediate retry), task deduplication (idempotency keys), monitoring (queue depth, processing time, failure rate). The right primitive for almost any non-trivial backend work.
When to use task queue
- Anything that doesn't fit in a single request lifecycle.
- Email, image processing, batch jobs, LLM calls.
Common mistakes
- No idempotency — retries process the same task twice.
- No dead-letter — failing tasks loop forever.
FAQ
What is task queue?
A task queue is the backend pattern where work units are submitted to a queue, picked up by workers, and processed asynchronously — Celery, BullMQ, Sidekiq, AWS SQS, Cloud Tasks, Trigger.dev, Inngest are 2026 examples.
When should I use task queue?
Anything that doesn't fit in a single request lifecycle. Email, image processing, batch jobs, LLM calls.
What are the most common mistakes with task queue?
No idempotency — retries process the same task twice. No dead-letter — failing tasks loop forever.
Related terms
- Background job — A background job is the unit of work in a [[task-queue]] system — a small piece of code that runs out-of-band from a request, may be retried on failure, and reports back via webhook / dashboard / database update.
- Durable execution — Durable execution is the workflow / agent pattern where long-running multi-step processes survive crashes, restarts, and queue failures — state checkpointed after each step, resume from last checkpoint. Inngest, Temporal, Restate, AWS Step Functions are 2026 implementations.
- Workflow engine — A workflow engine is the orchestration runtime — n8n, Make.com, Zapier, Temporal, Airflow — that executes multi-step business processes, handles retries, manages state, and integrates with external systems.
Last updated: 2026-06-01. Raw markdown: https://promtable.com/glossary/task-queue.md.