LLM Cost Tracking: Detect Token Spend Spikes by Customer

Last verified on Nov 8, 2026.

LLM spend behaves differently from most infrastructure spend. A server costs the same whether it is busy or idle. A model call costs per token, so one bug, one oversized prompt or one customer with an agent that will not stop can burn a week of budget in a day. The invoice arrives a month later, and by then the cause is gone.

OWASP lists this as a security risk too. Its LLM10

What the evidence looks like

Every model response reports its own usage. The Anthropic Messages API returns a usage object with input_tokens, output_tokens and cache fields such as cache_creation_input_tokens and cache_read_input_tokens. OpenTelemetry's generative AI conventions define matching attributes: gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.usage.cache_read.input_tokens, gen_ai.usage.cache_creation.input_tokens and gen_ai.usage.reasoning.output_tokens.

The evidence you need is one log event per model call carrying those counts plus who and what caused it:

{
  "ts": "2026-11-06T16:02:11Z",
  "event": "llm.call",
  "feature": "support-agent",
  "customer": "c_204",
  "run_id": "run_91",
  "step": 14,
  "model": "large-model",
  "input_tokens": 61800,
  "output_tokens": 420,
  "cache_read_input_tokens": 0
}

This is an example. Fourteen steps into one run, the input is 61,800 tokens. A runaway agent usually looks like that: the same run_id with a step count in the hundreds and an input size that grows on every step because the whole history is resent.

How to detect it step by step

1. Log every call with attribution. Capture feature, customer or tenant, run or conversation ID, step number, model, and every token count the provider returns. Without the attribution you learn that spend jumped, not why.

2. Turn tokens into money. Keep a small price table per model, with input, output and cached input rates, and compute cost per call in a query or a pipeline. Update the table when prices change.

3. Chart spend by feature and customer, hourly.

SELECT date_trunc('hour', ts) AS hour, feature, customer,
       sum(input_tokens * p.input_rate + output_tokens * p.output_rate) AS cost
FROM llm_calls c JOIN price_table p USING (model)
WHERE ts > now() - INTERVAL '7 days'
GROUP BY 1, 2, 3

4. Compare to a baseline per key. Alert when a feature or customer spends more than a multiple of its own trailing median for that hour of the week, and above an absolute floor so a tiny account does not alert for a few cents. Use a second rule on cumulative spend for the day against the daily budget.

5. Look for loop shapes directly. Runs with many steps, input tokens that rise each step and calls to the same tool with the same arguments:

SELECT run_id, customer, count(*) AS steps, max(input_tokens) AS max_input, sum(output_tokens) AS out_tokens
FROM llm_calls WHERE ts > now() - INTERVAL '1 hour'
GROUP BY 1, 2 HAVING count(*) > 50

6. Add a hard ceiling. Detection is late by design. A per-customer daily token cap and a per-run step limit stop a runaway automatically while the alert tells a person.

Thresholds and pitfalls

Start with a ratio of 3 times the trailing median for the same hour of the week, a floor of a few dollars, and a daily budget alert at 50% and 100%. Tune after a month.

Things that distort the numbers:

• Prompt caching. Cache reads are billed differently from fresh input. If you sum all input tokens as one price, a cache change looks like a spend change. Track the cache fields separately.
• Price changes and model swaps. A model migration moves cost with no change in usage. Include the model as a dimension and alert on token counts too.
• Retries. An upstream error followed by retries multiplies calls. Compare calls per user action with the baseline.
• Reasoning tokens. Models that think before answering can produce far more output than the visible reply suggests. Record the reasoning count when the provider reports it.
• Batch jobs. A planned backfill spikes spend legitimately. Tag them and exclude them from the per-customer alert, with their own budget.

Provider usage counts and your invoice will not match exactly. Use your logs to find causes and the provider's dashboard to reconcile the total.

What to do when it fires

Find the key with the largest delta by feature, customer and run. Stop the run if it is still going. A step limit and a token cap in the agent loop are the usual fix, along with trimming or summarizing history so the input stops growing. If a single customer is responsible, check whether the traffic is a bug on your side or abuse of an open endpoint.

Afterward, set budgets per feature and per customer, and make the daily ceiling a hard stop. Related guides: cloud spend anomalies, cost anomalies, rogue AI agents, agent misbehavior and LLM behavior drift. To collect call logs, see the Langfuse, OpenRouter and OpenTelemetry integrations.