All posts
Internet Fundamentals

Background Jobs and Queues: At-Least-Once Delivery, Visibility Timeouts and Dead Letters

By

October 26, 2026

6 min read

Anything slow, flaky or bursty belongs off the request path: email, webhooks, AI calls, exports, image processing. Queues make that safe, provided your workers expect the same message twice. How SQS works, the settings that matter, and the failure modes of jobs built with setTimeout and cron.

Internet Fundamentals
Queues
Amazon SQS
Background Jobs
Reliability

A request that sends an email, calls an AI model and resizes an image is slow and fails whenever any of those does. Put the slow parts on a queue, return immediately, and let workers do them with retries.

How a queue like SQS behaves

  • At-least-once delivery: a standard queue can deliver a message more than once. Workers must be idempotent, so a duplicate does no harm.

  • Visibility timeout: when a worker receives a message, it is hidden for a period. If the worker does not delete it before the timeout, it becomes visible again and another worker retries it. Set the timeout longer than the job takes.

  • Dead-letter queue: after a set number of failed receives, the message moves to a separate queue for inspection, instead of retrying forever.

  • Ordering: standard queues do not guarantee order. FIFO queues do, within a message group, at lower throughput.

Want a second opinion on this for your system?

Book a free 30-minute call. You'll leave knowing what we'd do first and roughly what it costs, whether or not you hire us.

Settings that matter

Setting

Rule of thumb

Visibility timeout

Several times the job's typical duration; for Lambda consumers, at least six times the function timeout, as AWS recommends

Max receives before dead-letter

3 to 5 for most jobs

Message retention

Long enough to fix a bug and replay, such as 4 to 14 days

Batch size

Larger for cheap jobs; 1 for slow or expensive ones, so one failure does not retry a batch

Alarms

On dead-letter queue depth above zero and on the age of the oldest message

Why not setTimeout or a cron job on the web server

  • Work in memory is lost on every deploy, crash or scale-in.

  • Two web servers run the same cron job twice.

  • There is no record of what failed and no way to replay it.

hoolco Monthly Review

Once a month: what changed in tech that month and what it means for you, from cloud and AI to security and the tools you build with. Every item links to its source. No sales pitch, unsubscribe in one click.

As of October 26, 2026. Provided as is, for general information, and accurate to the best of our knowledge on the date shown. Not legal, compliance, tax, financial or other professional advice. Prices, limits and vendor services change; check current vendor documentation before relying on anything here. Terms of use.