Third Party Monitoring: Detect Vendor and API Degradation

Last verified on Nov 8, 2026.

Your product is only as reliable as the slowest service it calls. Payments, email, authentication, maps, search and model providers all sit in your request path. When one of them gets slow or starts returning intermittent errors, your own p95 climbs, queues back up and users see timeouts, often before the vendor admits anything is wrong.

Third-party degradation is easy to misread. Your dashboards show your service slow, and the first hour of an incident goes to your own code. The way out is to measure every outbound dependency as if it were a service of your own, with its own latency, error rate and retry count.

What the evidence looks like

Evidence comes from the calls you make, not from the vendor's status page. One log line per outbound request is enough:

{
  "ts": "2026-11-07T13:31:20Z",
  "vendor": "payments",
  "operation": "charge.create",
  "status": 200,
  "duration_ms": 2840,
  "attempt": 2,
  "timeout_ms": 5000
}

This is an example. A normal call for this operation might take 300 ms on the first attempt. Here it took 2.8 seconds on the second attempt, so the first failed or timed out too. Across the fleet that looks like vendor latency doubling, a rising share of retries and timeouts clustered on one operation or one region.

The Google SRE book names latency, traffic, errors and saturation as the four golden signals of monitoring, and the same four apply to a dependency. OpenTelemetry defines a standard histogram, http.client.request.duration, for the duration of outbound HTTP requests, which gives you the first signal without custom code.

How to detect it step by step

1. Instrument every outbound call. Record vendor, operation, status, duration, attempt number and whether the call timed out. A shared HTTP client wrapper is the right place. If you already emit http.client.request.duration, add the vendor as a label based on the host.

2. Compute per-vendor latency and error rate.

histogram_quantile(0.95,
  sum by (le, vendor) (rate(http_client_request_duration_seconds_bucket[5m])))

sum by (vendor) (rate(http_client_request_duration_seconds_count{http_response_status_code=~"5.."}[5m]))
  / sum by (vendor) (rate(http_client_request_duration_seconds_count[5m]))

The exact metric names depend on your exporter, so check what yours emits. The point is the grouping by vendor.

3. Compare each vendor to itself. A vendor that normally takes 80 ms and now takes 200 ms is degraded even though 200 ms looks fine in absolute terms. Use the same window from the previous week and a ratio.

SELECT vendor, operation,
       percentile_cont(0.95) WITHIN GROUP (ORDER BY duration_ms) FILTER (WHERE ts > now() - INTERVAL '10 minutes') AS p95_now,
       percentile_cont(0.95) WITHIN GROUP (ORDER BY duration_ms) FILTER (WHERE ts BETWEEN now() - INTERVAL '7 days 10 minutes' AND now() - INTERVAL '7 days') AS p95_last_week,
       avg((attempt > 1)::int) FILTER (WHERE ts > now() - INTERVAL '10 minutes') AS retry_share
FROM outbound_calls GROUP BY 1, 2

4. Probe from the outside as well. Prometheus's blackbox exporter probes endpoints over HTTP and other protocols and reports probe_success and timing metrics. A synthetic call that runs once a minute covers low-traffic vendors where real calls are too rare to measure.

5. Read the vendor's status feed, but do not wait for it. Statuspage has an API with endpoints for incidents, so if a vendor publishes there you can pull open incidents and show them next to your own graphs. Status pages lag reality, so use them for confirmation and customer messaging, not as the trigger.

Thresholds and pitfalls

Alert on a sustained change, for example p95 above 2 times last week's value for 10 minutes or an error ratio above 5% for 5 minutes, and require a minimum number of calls in the window so a quiet vendor does not page on three slow requests.

Things that cause false alarms or missed alerts:

• Your own network. A slow NAT gateway, DNS or an exhausted connection pool looks like vendor trouble. Check whether many vendors degrade together; if so, look at your side.
• Per-operation differences. A report endpoint is slow by design. Group by operation so heavy calls do not hide behind light ones.
• Client timeouts. Calls cut off at your timeout all show the same duration. Count timeouts as a separate series.
• Retry amplification. Retries multiply load on a struggling vendor and on your workers. Track attempts per call and cap them.
• Regional or account scope. Some incidents affect one region or one plan. Include region as a label if the vendor offers it.

What to do when it fires

Confirm it is the vendor: compare against other vendors and your own services, and try a direct call from outside your network. Then protect yourself. Lower timeouts so slow calls free workers sooner, open a circuit breaker to fail fast, serve cached or queued results where the business allows it and stop retrying immediately.

Contact the vendor with request IDs and timestamps from your logs. Afterward, record the duration of the incident as seen from your side, since that is the figure you will need for a service credit conversation. Related guides: webhook gaps covers a vendor that stops sending, third-party failures covers the wider group and latency spikes covers the effect on your own endpoints. Integration pages for common dependencies include Stripe, Twilio and Prometheus.