Tail Latency Alerts: An Issue When the Slowest Requests Get Slower

Last verified on Nov 10, 2026.

What it catches

An average hides slow requests. If 99 requests in 100 finish in 120 ms and one takes six seconds, the median looks perfect and one user in a hundred is waiting. Tail latency is that slowest slice. It tends to worsen first, and it worsens for reasons the median never shows:

• One large account has a query that scans far more rows than anyone else's.
• A lock or garbage collection pause stalls a few requests at a time.
• A slow replica receives a share of reads.
• A retry inside the handler doubles the time for the unlucky calls.

The Sazabi agent sets up sensors for your project. You set nothing up: the slowest-requests question is asked for each route.

Example: invoices export with a flat median

This is an example, not output from a real project. A billing-api service logs one line per request, with the account it served:

2026-11-07T11:02:14Z INFO billing-api GET /v1/invoices status=200 duration_ms=118 account=acct_2041
2026-11-07T11:02:15Z INFO billing-api GET /v1/invoices status=200 duration_ms=131 account=acct_0877
2026-11-07T11:02:19Z INFO billing-api GET /v1/invoices status=200 duration_ms=6240 account=acct_9930
2026-11-07T11:02:22Z INFO billing-api GET /v1/invoices status=200 duration_ms=122 account=acct_1522
2026-11-07T11:02:31Z INFO billing-api GET /v1/invoices status=200 duration_ms=5912 account=acct_9930

Most lines look like the ones that always looked fine. The two slow ones share an account. A median-based view never moves. An example issue:

• Name: Slowest GET /v1/invoices requests taking several seconds
• Description: The slowest requests on this route are far slower than usual while the typical request is unchanged. The slow ones cluster on one account.
• Severity: medium
• Component: billing-api

The point of the issue is the clustering. The analysis names the shared factor it found in the evidence instead of handing you a percentile chart.

How Sazabi sets it up

The Sazabi agent sets up sensors for your project and maintains them over time. For this signal the question, in plain words, is: "Tell me when the slowest requests on a route are much slower than usual." Each route is judged on its own, so one slow route does not hide behind healthy ones.

A regular daily peak is part of normal behavior and does not count. A real shift does.

You do not choose a percentile, a window or a threshold. If you want something narrower, such as a stricter bar for one critical route, you can ask support for a specific sensor in plain words.

What you see

A firing starts an investigation. A real problem becomes an issue with its description, severity and evidence attached. For this kind of issue, expect the evidence to focus on what the slow requests have in common. Ask the agent in the thread to compare the slow requests with the fast ones on any field your logs carry: account, region, payload size, upstream host.

Delivery rules can route this to the team that owns the component, so a platform team is not woken for a billing query.

What you need

A duration on each request log line. Access logs and OpenTelemetry spans both work. Extra fields such as an account or a region make the investigation sharper. Structured logging gives you those fields for free. Sources are listed under integrations.

What it does not cover

• A change in typical latency. See latency step changes and latency drift.
• Latency you cannot see because no duration is logged.
• Percentile math on data Sazabi does not receive, such as spans you sampled away.
• Slow work that is not a request, like a nightly job.

The by-hand method with histograms and ratios is in the tail latency guide. For definitions, see latency percentiles. Issues Sazabi detects are metered, so check pricing.