Monitoring AI agents in production: what to log, alert on and review

Ordinary software tends to fail loudly. A server goes down, an error appears, someone gets paged. AI agents fail quietly. The agent keeps replying, politely and confidently, while it gives an outdated price, skips a step it used to take, or starts handing every third customer to a person. Nothing crashes, so nothing alerts.

AI agent monitoring means recording what the agent sees, decides and does in every conversation, and checking those records for problems before customers report them. Testing before launch tells you the agent was good on the day it went live. Monitoring tells you it is still good today.

#Why agents drift after launch

• The world changes. Prices, policies, stock and opening hours change, and the documents or data the agent reads may not keep up.
• Customers change. A new product, a promotion or a delivery problem brings questions the test set never covered.
• The model changes. Providers update models, sometimes behind the same name, and behaviour shifts in small ways.
• Connected systems change. A CRM field is renamed, an API slows down, a password expires.
• The agent changes. Every prompt edit or new tool can fix one thing and break another.

#What to log for every conversation

The basic unit of monitoring is a trace: one record per conversation, holding every step the agent took. Without traces, you can see that something went wrong but not why.

conversation: c-48213
channel: whatsapp
versions: { prompt: v14, model: "provider-model-2026-09-01", tools: v6 }
steps:
  - customer: "Order A-10XX still not here, it's been a week"
  - tool: get_order
    input: { order_id: "A-10XX" }
    output: { status: "with courier", eta: "yesterday" }
    ms: 420
  - tool: get_courier_status
    error: timeout
    ms: 8000
  - agent: "Sorry, I can't reach the courier right now. I've asked our team to check and they'll message you today."
  - handover: { to: support_queue, reason: tool_failure }
outcome: handed_over
cost_usd: 0.004
total_ms: 11200

At minimum, record: the messages, the documents retrieved, each tool call with its inputs, outputs, errors and timing, the versions of the prompt, model and tools, the cost, and the outcome (resolved, handed over, abandoned). Version numbers matter more than they look: they are how you connect a change in behaviour to the change that caused it.

#Alerts: what needs attention now

Alert only on things that need someone to act within hours. Too many alerts, and people stop reading them. These are the ones we set up:

Every alert needs a named owner and a first step, such as "check the CRM connection" or "pause refunds". The most important first step is the kill switch: a way to switch the agent off, or limit it to answering questions only, in seconds. See guardrails for AI agents.

#Trends: what to review weekly

Slower changes do not need an alert, but they do need a regular look. A weekly dashboard of the production metrics from AI agent evaluation metrics covers most of it: resolution rate, human correction rate, escalation rate, response time, cost per resolved task and customer feedback. Read them as trends. One bad day is noise; four weeks of slow decline is a signal.

#The review queue

Numbers show that something changed. Reading conversations shows what. We send these conversations to a review queue for a person to read:

• Every handover, to check it was needed and that the summary was useful.
• Unhappy customers: low ratings, complaints, or angry language.
• Staff corrections: drafts that were edited heavily, actions that were undone, tickets that were reopened.
• Low grader scores, if a second model checks a sample of answers for accuracy.
• A small random sample, because the worst failures are often the ones nobody flagged.

Each real failure gets fixed, and then becomes a new case in the test set, so the same mistake cannot quietly come back. This loop is what makes an agent better over time instead of slowly worse.

#Catching drift on purpose

Some changes happen without anyone touching the agent. Two habits catch them:

• Pin model versions where the provider allows it, and upgrade on purpose, after the test set passes on the new version.
• Re-run the full test set on a schedule, weekly for example, even when nothing changed. If yesterday’s pass becomes today’s fail with no change on your side, something underneath moved.

#A simple weekly routine

1. 01

   Check the dashboard

   Resolution, corrections, handovers, response time and cost, compared with the previous weeks.

2. 02

   Read the queue

   Go through flagged conversations and the random sample. Note each failure and its cause.

3. 03

   Fix and add tests

   Fix the causes, add each failure to the test set, and run the full set before releasing the fix.

4. 04

   Update the sources

   Confirm that prices, policies and documents the agent relies on are still current.

In the first weeks after launch this takes the most time, because that is when the agent meets the questions nobody predicted. It usually settles into a short weekly check once the common failures are fixed.

We build tracing, alerts and a review queue into every AI agent and chatbot we deliver. For the testing that comes before all of this, start with how to test an AI agent before go-live.