Monitoring AI in production: drift, cost and failures
A tested AI system can degrade without anyone changing it. What to watch once it is live, and the monthly routine that keeps it honest.
The short answer
An AI system that cleared its evaluation on launch day can be worse three months later without anyone having changed a line. The provider updates the model. Customers start asking about things the documents do not cover. A new kind of input appears. Volume grows past a rate limit. None of this produces an error message. Monitoring an AI system therefore means watching quality, not just uptime: sampling live cases and scoring them, tracking escalations and corrections, counting guardrail firings, and keeping cost and latency per case in view. A monthly routine catches drift while it is still small.
The six things to watch
| Signal | What it tells you | How |
|---|---|---|
| Quality on a live sample | Whether accuracy has drifted from launch | Score a random sample of live cases monthly against the evaluation standard |
| Escalations and corrections | Whether people are having to fix more | Rate per week, by category |
| Guardrail firings | What the system tried to do that was stopped | Count and examples, weekly |
| Cost per case and total | Efficiency and anomalies | Dashboard with caps and alerts |
| Latency | Whether people are waiting longer | Percentiles, not averages |
| Provider errors and rate limits | Whether calls are failing or being throttled | Error rate, alert on spikes |
The monthly routine
- Pull a random sample of live cases, including some escalations and some straight-through.
- Score them the way the evaluation set was scored, by the same kind of person.
- Compare with launch and last month, by category.
- Read the guardrail log and the corrections: what patterns are new.
- Check the model version the provider is serving; if it changed, rerun the full evaluation set.
- Review cost per case and total against budget; investigate any step change.
- Refresh the documents the system answers from where the sample showed stale citations.
- Add the month’s interesting cases to the evaluation set.
Cost control from day one
Set a daily and monthly cap at the provider. Add per-user and per-endpoint rate limits in your application. Alert on cost anomalies, not just totals. Log input sizes, because one user pasting whole documents changes the arithmetic. Review cost per case monthly; a rising number with steady volume means inputs grew or a model tier changed. Most AI cost stories are a loop or a bot that ran for a weekend, and a cap turns that into a small number.
What this means for you
Launching an AI system is the start of watching it. Sample and score live cases monthly, track escalations, corrections and guardrail firings, keep cost and latency per case visible, and rerun the evaluation whenever the provider changes the model. Set caps before launch. The systems that stay useful for years are not the ones that tested best on day one; they are the ones someone kept measuring.
Frequently asked questions
The system passed its tests. Why would it get worse?
Because the world around it changes while it stands still. The provider ships a new model version that behaves slightly differently. Customers start asking about a product that did not exist when the documents were indexed. A new type of input appears. Volume triples and a rate limit starts rejecting calls. Each is invisible unless something is watching.
What does drift look like in practice?
A slow rise in cases a person had to correct, more guardrail firings for a category that used to pass, answers citing documents that have been superseded, or an accuracy score on the monthly sample that has slipped a few points from launch. Any one of them is a prompt to rerun the evaluation set and look at what changed.
How do AI costs get out of hand?
Rarely from normal use. Usually from a loop that calls the model repeatedly, a bot hammering a public endpoint, inputs far larger than expected such as whole documents pasted in, or a model switch to a more expensive tier nobody priced. Caps per day and per user, alerts on anomalies and a look at cost per case monthly catch all of them.