Prompt Injection Detection and Agent Monitoring

Last verified on Nov 8, 2026.

An AI agent misbehaves when it does something its owner did not intend: it follows instructions hidden in a web page, calls a tool it was never meant to use, loops until a budget runs out or takes an action with more authority than the task needed. The output can look fine while the behavior behind it is not.

Detection works on what agents do, which means tool calls, arguments, loops and token use, rather than on what they say.

The failure modes

OWASP's Top 10 for LLM applications gives you the vocabulary. Prompt injection (LLM01) occurs when prompts alter the model's behavior or output in unintended ways. Indirect prompt injection is the version where the model accepts input from external sources such as websites or files, and the content changes how the model behaves. The input does not need to be human-readable, as long as the model parses it.

Excessive agency (LLM06) is about what the agent is allowed to do. OWASP names the root causes as excessive functionality, excessive permissions and excessive autonomy. An agent that only needed to read documents but also holds a delete tool is the standard example.

You rarely detect the injection itself. You detect the behavior it causes.

Evidence in telemetry

OpenTelemetry's GenAI semantic conventions, which are still in development status, define a span for tool execution. Its gen_ai.operation.name should be execute_tool, its name follows execute_tool {gen_ai.tool.name}, and opt-in attributes carry the tool call arguments and result. The inference conventions record token usage through gen_ai.usage.input_tokens. If you trace your agents with these attributes, the evidence is already in your trace store.

An example of a run that deserves a look:

run_id=a-3391 agent=support-bot step=2 tool=search_kb         args_len=64   ok
run_id=a-3391 agent=support-bot step=3 tool=fetch_url         url=https://untrusted.example.test/page ok
run_id=a-3391 agent=support-bot step=4 tool=send_email        to=external@example.test  ok

Support bots read knowledge-base articles. This one fetched an external page and then sent an email outside the company, a sequence it has never produced before.

Detect it step by step

1. Log every tool call with agent, run, tool name, argument summary, result status and the trust level of the data it touched. Avoid logging full prompts by default. The OWASP Logging Cheat Sheet lists sensitive personal data and secrets among things to exclude from logs.
2. Build a per-agent tool allowlist from history, then flag calls outside it:

-- Example: tools an agent used today but not in the previous 30 days
WITH history AS (
  SELECT DISTINCT agent, tool_name
  FROM tool_calls
  WHERE ts >= current_date - 30 AND ts < current_date
)
SELECT t.agent, t.tool_name, count(*) AS calls, min(t.ts) AS first_call
FROM tool_calls t
LEFT JOIN history h ON h.agent = t.agent AND h.tool_name = t.tool_name
WHERE t.ts >= current_date AND h.agent IS NULL
GROUP BY t.agent, t.tool_name;

3. Flag risky sequences, not only risky tools: untrusted content ingested, followed by a tool that sends data out or changes state, in the same run.
4. Detect loops: the same tool with the same arguments more than a handful of times in one run, or step counts far above the agent's usual range.
5. Watch cost per run. Input tokens per run that triple after a prompt or tool change usually mean a loop or runaway context. See LLM spend spikes and the cost anomalies guide.
6. Compare outcomes across versions. A prompt or model change can alter tool-use frequency; see LLM behavior drift.

Thresholds and pitfalls

Keyword filters for phrases like "ignore previous instructions" catch only crude attempts and are easy to evade. Use them as one weak signal and prefer behavior rules.

• A legitimate new tool will look like an anomaly the first day. Update the allowlist as part of the release that adds it.
• Rare but valid tasks hit tools once a quarter. Use a long history window.
• Tool-call arguments may contain user data, so apply the same retention and access rules you use elsewhere.
• Allowlists defend against drift, not against an agent abusing a tool it is allowed to use. For that, limit scopes and rates.

What to do when it fires

Stop the run and revoke or pause the agent's credentials if outbound data is involved. Review the full trace to find where the untrusted content entered. Then reduce authority: remove tools the task does not need, scope credentials to the minimum and require human approval for actions that send data out or are hard to undo.

For agents with broader reach, read rogue AI agents. The wider security monitoring guide places this among other threats. Telemetry setup is covered by OpenTelemetry and the AI frameworks category.