Background Job Failure Alerts: Issues for Missed Runs, Retry Loops and Growing Backlogs

Last verified on Nov 10, 2026.

Work that nobody is watching

Background work runs when no person is looking. A cron job skips a night. A queue consumer dies and the queue keeps growing. One poisoned record gets retried forever. Each of these is invisible from the web app, because the web app keeps answering.

What the agent watches for

Every job system is different, so what gets watched is written for your project. The Sazabi agent reads your logs to find the jobs, queues and schedules in them, then sets up sensors for each and keeps them current. Six signals feed this page:

• Missed schedule. A cron job did not run, ran twice or ran long.
• Retry storm. The same operation retried in tight loops, such as one job retried many times for a single record.
• Backlog growth. A queue grows faster than it drains.
• Queue wait growth. Work waits longer before it starts, so an email goes out long after its trigger.
• Empty output. A job finishes with success and produced nothing.
• Stale data. A cache or sync job stopped refreshing.

Each job is judged on its own, so a failing weekly export does not hide inside a healthy hourly cleanup.

Example: one record retried to exhaustion

This is an example, not output from a real project. A worker called invoice-sync pulls records from a queue:

2026-11-08T22:10:01Z WARN invoice-sync record=inv_5521 attempt=1 error="vendor timeout"
2026-11-08T22:10:02Z WARN invoice-sync record=inv_5521 attempt=2 error="vendor timeout"
2026-11-08T22:10:02Z WARN invoice-sync record=inv_5521 attempt=3 error="vendor timeout"
2026-11-08T22:10:03Z WARN invoice-sync record=inv_5521 attempt=4 error="vendor timeout"

The attempts come a second apart, and the counter keeps climbing. Each line is a warning, which is why it is easy to scroll past, and the loop burns worker time and calls the vendor each pass. The issue could read:

• Name: invoice-sync retrying one record without backoff
• Description: inv_5521 has been retried many times in a few seconds, each time failing on a vendor timeout. Other records are waiting behind it.
• Severity: medium

The related sensor, Backlog growth, may also fire a little later as the queue behind the stuck record builds. The agent investigates what fired and decides what to open.

Seeing it and acting on it

You see an issue with a name, severity, description, the analysis of why, and evidence. Ask the agent follow-up questions in the thread, such as whether other jobs share the vendor. Use sazabi issues agent-prompt to hand the fix to a coding agent. Resolve the issue after the fix; if the job fails the same way later, a new issue opens, because a recurrence of a resolved issue is a new one.

For routing, use a delivery rule on component or owning team. The data team can take every job issue, while paging stays reserved for high and critical ones.

What it needs

• A line when the job starts, finishes or fails, with a job name field.
• For queues, a depth or wait figure in logs, or OpenTelemetry metrics sent to the project.
• For durable-execution platforms, the log sources in the durable execution category.

What it does not cover

• A job that logs nothing at all when it runs. The sensors cannot tell skipped from never-instrumented.
• Jobs inside systems whose logs you do not send to Sazabi.
• Running the job again for you. The issue gives the analysis and the next step.
• Duplicate processing, which is not covered today.

Related

The missed schedules guide explains the same problem without Sazabi, and the stale data guide covers the sync case. For capacity side effects, see capacity risks, and for jobs that complete without doing anything, silent failures. Issues detected are metered; see pricing.