Mustlist

Background jobs and queues for a Next.js + Postgres app

VerifiedLast verified 15 Nov 2026R@RIDAM7v1.0.0
Suggest a change

Enqueue each job in the same Postgres transaction as its write and run it in a separate long-running worker (pg-boss or Graphile Worker). Make handlers idempotent, retry with capped backoff, keep failed jobs to replay, and never treat after() as a queue.

Forweb-appsaase-commercemarketplaceinternal-tool
Not forstatic-site-without-backendhigh-volume-event-streaming

Must-have checklist

12 must · 3 should

Must

01

Never use after(), waitUntil or an un-awaited promise for work that must happen; enqueue a durable job instead.

`after()` runs only within the route's max duration and is never retried, so a crash, deploy or timeout loses the work.

Data
02

Enqueue the job in the same Postgres transaction as the business write (pg-boss db option, graphile_worker.add_job() or an outbox row), never in a separate call.

Enqueue-then-rollback acts on data that never existed; commit-then-crash silently loses the job.

Data
03

Make every handler safe to run twice: guard each side effect with a unique constraint, an upsert or a provider idempotency key derived from the job's business id.

Every option here delivers at least once; a worker can die after the side effect but before marking the job done.

Data
04

Retry with exponential backoff plus jitter, cap the delay, and set max attempts per queue from how long the work stays useful; fail fast on errors that cannot succeed.

pg-boss defaults to 2 retries with no delay; Graphile Worker defaults to 25 attempts over about 3 days. Neither fits every job.

Performance
05

Keep permanently failed jobs in an inspectable failed or dead-letter state with the last error, alert on them, and provide a replay path after a fix.

pg-boss deletes failed jobs after deleteAfterSeconds (7 days by default); Vercel Queues has no dead-letter queue and expires messages.

Data
06

Set a job lease above the worst-case run time and send heartbeats for long jobs, so a crashed worker's job returns to the queue quickly.

A killed Graphile Worker leaves its jobs locked for about 4 hours; pg-boss expires active jobs after 15 minutes by default.

Performance
07

In a hand-rolled jobs table, claim a batch with FOR UPDATE SKIP LOCKED in a short transaction that sets locked_until, then do the work outside that transaction.

Workers never block each other, and no row lock or connection is held during slow I/O.

Performance
08

Run each scheduled occurrence once across instances: use the queue library's cron, or take pg_try_advisory_xact_lock and record the scheduled time under a unique key.

Vercel documents that a cron run can be delivered twice and that runs can overlap.

Data
09

Run long-lived workers outside serverless functions; a serverless consumer or cron drain must stop claiming well before maxDuration and leave the rest queued.

Vercel stops a function at its max duration (300 s default; 800 s maximum on Pro and Enterprise).

Performance
10

On SIGTERM stop fetching new jobs, let in-flight jobs finish within a timeout, then exit; make the platform's shutdown grace period longer than that timeout.

Jobs cut off mid-run wait out their lease and may repeat side effects.

Performance
11

Authenticate every route that starts work: check the CRON_SECRET bearer token, verify managed-service signatures on the raw body, and exclude these routes from proxy.ts redirects.

Security
12

Track queue depth, age of the oldest ready job, failure rate and dead-letter count per queue, and alert on oldest-job age, not depth alone.

A stopped worker shows up as rising age; pg-boss reports readyOldestSeconds for each queue.

Performance

Should

13

Give jobs that must exist once per entity a deterministic key (pg-boss singletonKey with a matching policy, Graphile job_key), and still enforce the outcome with a constraint.

Queue keys only dedupe jobs that are still pending; completed Graphile Worker jobs are deleted and their keys freed.

Data
14

Put ids in job payloads, not secrets, tokens or full personal data; load current data and re-check permissions in the handler.

Payloads are kept for days in job tables, dashboards and third-party services, and data can change before the job runs.

Security
15

For user-started jobs such as exports, store a status row the UI can poll, show failures with a retry action, and notify the user when the result is ready.

UX

Decisions to make

Asked before building

01Can you run a long-running process (container, VM or Vercel Daemons) next to the Next.js app?

02How soon must a job start after it is enqueued?

03How long can the longest job run?

04Do jobs have several steps with long waits (hours or days) or wait for outside events?

05May job payloads be stored by a third-party service?

Options

4 compared
RecommendedFits your answers

Postgres-backed queue with a separate worker process

Examples, not endorsements: pg-boss, Graphile Worker

Best fit when
  • You can run one always-on process next to the app
  • You want jobs enqueued in the same transaction as your writes
  • You want no extra vendor and no per-job fee
Avoid when
  • You can only deploy serverless functions
  • Job volume is high enough to strain the primary database
Trade-offs
  • You deploy, scale and monitor the worker yourself
  • Job rows and polling add storage and query load to Postgres
  • Each worker instance holds database connections

Managed durable job service

Examples, not endorsements: Inngest, Trigger.dev, Upstash QStash, Vercel Queues (beta)

Best fit when
  • You deploy only to serverless functions
  • Jobs have many steps, sleeps or waits for outside events
  • You want hosted retries, replays and a dashboard
Avoid when
  • Payloads must not leave your infrastructure
  • Jobs must commit atomically with your write and you cannot add an outbox relay
Trade-offs
  • Enqueueing is a network call outside your transaction, so write an outbox row and relay it, or reconcile
  • Per-run, per-step or per-operation fees on top of function compute
  • Dedupe and retention windows are set by the vendor (QStash 10 minutes, Inngest 24 hours)

Cron-triggered drain of a jobs table

Examples, not endorsements: Vercel Cron Jobs, Vercel Schedules (beta)

Best fit when
  • Low volume where a delay of a minute or more is acceptable
  • Serverless-only hosting with no budget for another service
Avoid when
  • Jobs must start within seconds
  • One job can run longer than the function max duration
  • You are on Vercel Hobby, where Cron Jobs run at most once a day
Trade-offs
  • You build claiming, leases, retries and dead-lettering yourself
  • Vercel does not retry a failed cron run; the next tick must pick up leftovers

Inline after() for non-critical work

Best fit when
  • Analytics, logging or cache warming that may be lost
  • Nudging a durable worker right after enqueueing
Avoid when
  • Any work that must happen, or must not happen twice
Trade-offs
  • No persistence and no retry; lost on crash, deploy or timeout
  • Bounded by the route's max duration

Why this approach

Jobs stored in the same Postgres database as your data can be enqueued in the same transaction as the write they belong to, so a job exists only if the write commits. A Postgres-backed library such as pg-boss or Graphile Worker adds retries with backoff, leases, cron and SKIP LOCKED claiming without another vendor, and matches the event table the payment-webhooks card drains. It needs a long-running worker process, which a serverless function cannot host, so run the worker on a container host, a VM or Vercel Daemons (beta). Choose a managed durable service when you can only deploy serverless functions or need multi-step flows with long waits, and a cron-triggered drain only for low-volume work that can wait a minute or more.

Lenses

Every route that starts work is public unless you lock it: cron routes, queue consumers and managed-service callbacks. Compare the Authorization: Bearer header with CRON_SECRET (each deployment keeps the value it was built with), and let the managed SDK verify signatures on the raw body (QStash Receiver, Inngest INNGEST_SIGNING_KEY). Keep these paths out of proxy.ts auth redirects. Put ids in payloads, never tokens, passwords or full personal data: payloads sit in job tables, logs, dashboards and vendor storage for days. Load current data and re-check permissions in the handler, because a user can lose access between enqueue and run. Never let user input choose the task name or queue.

Version notes

pg-boss>=12.0.0 <13.0.0
  • v12 has no default export (import { PgBoss } from 'pg-boss') and needs Node.js 22.12+ and Postgres 13+. Call createQueue() before send() or work().
  • Pass { db: fromDrizzle(tx, sql) } (or fromPrisma, fromKysely, fromKnex) to send() so the job commits with your transaction.
  • Defaults are retryLimit 2 with no delay or backoff and expireInSeconds 900; set them per queue, plus heartbeatSeconds for long jobs.
  • Failed jobs are deleted after deleteAfterSeconds (7 days); add a deadLetter queue and use redrive() to replay.
graphile-worker>=0.18.0 <0.19.0
  • 0.18 is ESM-only, needs Node.js 22.18+ and reads graphile.config.ts. Enqueue atomically with graphile_worker.add_job(...) inside your transaction.
  • max_attempts defaults to 25 (about 3 days of backoff); failed jobs stay until you reschedule or delete them.
  • job_key is not an idempotency record: completed jobs are deleted, and a running job can get a second copy.
Official docsContext7worker.graphile.org
next>=16.0.0 <17.0.0
  • after() runs within the route's max duration with no persistence or retry; use it to nudge a durable worker, never as a queue.
  • Set export const maxDuration in cron and consumer Route Handlers; the platform stops the function there.
inngest>=4.0.0 <5.0.0
  • v4 moves triggers into the options, createFunction({ id, triggers: { event: 'app/x' } }, handler), and defaults to cloud mode (INNGEST_DEV=1 locally).
  • Each step.run() retries on its own (4 retries by default), so give each side effect its own step; event idempotency lasts 24 hours.
Official docsContext7www.inngest.com
@upstash/qstash>=2.0.0 <3.0.0
  • Verify Upstash-Signature on the raw body with Receiver or verifySignatureAppRouter.
  • 3 retries by default, then a dead-letter queue; publish deduplication lasts only 10 minutes.

Verify checks

12 checks
01

When the business transaction rolls back after enqueueing, no job exists; when it commits, exactly one job exists.

integrationData
02

A handler killed after its side effect but before completion is retried, and the side effect (for example one sent-email row) happens once.

integrationData
03

Enqueueing the same logical job twice, including once after the first has completed, produces one side effect.

integrationData
04

A handler that always throws is retried with growing delays up to the max attempts, then ends in a failed or dead-letter state with its last error.

integrationPerformance
05

A failed job can be replayed after a fix and completes exactly once.

integrationData
06

A job whose worker process is killed is picked up by another worker within the configured lease or heartbeat window.

integrationPerformance
07

Two workers polling the same queue never process the same job at the same time.

integrationPerformance
08

With two worker instances running, a scheduled job runs once per occurrence, and a duplicate trigger for the same scheduled time does nothing.

integrationData
09

A request to a cron or consumer route without a valid CRON_SECRET or signature gets 401 and enqueues or runs nothing.

integrationSecurity
10

Sending SIGTERM to a worker mid-job lets the job finish or return to the queue, and the worker fetches no new jobs.

integrationPerformance
11

A cron drain facing a large backlog stops claiming before its time budget runs out and leaves the remaining jobs queued for the next run.

integrationPerformance
12

Stopping all workers triggers the oldest-ready-job age alert within its threshold.

manualPerformance

Common mistakes

  • Treating after(), waitUntil or an un-awaited promise as a queue, so work disappears on a crash, deploy or timeout.
  • Enqueueing before commit (a job for a rolled-back write) or after commit in a separate call (a lost job if the process dies in between).
  • Assuming exactly-once delivery, then sending an email or charging a card again on retry.
  • Letting retention delete failed jobs before anyone has looked at them.
  • Starting a polling worker inside a Route Handler or instrumentation.ts on serverless hosting, where instances stop at the max duration or between requests.
  • Shipping a cron route without the CRON_SECRET check, or letting overlapping runs process the same rows.
  • Killing workers with SIGKILL on deploy, which leaves jobs locked until their lease expires.
  • Alerting on queue depth only, so a stopped worker with a small backlog goes unnoticed.

Sources

25 sources