Background Jobs Without a Worker Fleet: SQS + EventBridge Scheduler

Almost every API grows a need to do things outside the request: send the email, generate the export, run the nightly reconcile. On EMPATHIKA I split that into two distinct questions — “do this now but not inline” and “do this later, on a schedule” — and AWS has a clean answer for each: SQS and EventBridge Scheduler. Neither requires a worker process you keep alive and watch.

Queue-driven side effects with SQS

When a request triggers slow work, the API should not wait for it. It enqueues a message and returns; a consumer does the work:

await sqs.sendMessage({
  QueueUrl: env.JOBS_QUEUE,
  MessageBody: JSON.stringify({ type: 'invoice.generate', id }),
});
// request returns now; the PDF is built off the hot path

The user-facing action stays fast, and a slow or flaky downstream never blocks it. SQS also hands you retries and a dead-letter queue with no code: a message that keeps failing is redelivered a few times, then parked in the DLQ for inspection instead of vanishing.

Two non-negotiables for consumers

• Idempotency. SQS is at-least-once, so a handler will occasionally see the same message twice. Dedupe on a job id before acting — generating one invoice twice is a bug, not a retry.
• Visibility timeout tuned to the work. While a consumer holds a message, it is invisible to others. Set the timeout above the job’s worst-case duration, or a long job gets picked up again mid-flight.

Scheduled work with EventBridge Scheduler

The other half is time-based: reminders, periodic syncs, cleanups. Instead of an always-on cron process (one more thing to deploy, scale and monitor), EventBridge Scheduler is managed cron — you register a schedule and a target, and AWS invokes it when due.

await scheduler.createSchedule({
  Name: `reminder-${appointmentId}`,
  ScheduleExpression: 'at(2026-04-20T09:00:00)',
  Target: { Arn: jobsQueueArn, RoleArn: invokeRole, Input: payload },
  FlexibleTimeWindow: { Mode: 'OFF' },
});

A nice pattern falls out of this: have the scheduler drop its event onto the same SQS queue. Then scheduled and triggered work flow through one consumer with one set of retry and idempotency guarantees — one code path to reason about, not two.

Why this beats a worker fleet

No idle worker process, no cron box, no custom retry logic — AWS owns the durability, the redelivery and the timing. Your API stays a request/response service, and all the asynchronous and scheduled work lives in two managed primitives. Less to run, less to break.