How do I detect silent model drift between AI runs using recorded baselines and replayed inputs?

What should I check, step by step?

1. Pick 50 to 500 canary inputs from real traffic, covering the tasks you care about, with inputs and expected properties frozen and hashed. Don't tune prompts on them.

2. Pin everything you control: model ID, settings, prompt template commit, tool schemas, retrieved context (store it, don't re-retrieve). Drift you introduce yourself isn't model drift.
3. Record a baseline run: for each item the output hash, the reported model value, system_fingerprint or modelVersion, deterministic check results (JSON schema valid, required fields present, no secrets, length) and a task score. Seal it before analysing.
4. Run each item several times (for example 5) in the baseline, so you know the normal spread: exact-match rate between repeats and the natural variance of the score.
5. Replay the same inputs on a schedule (daily, and after any provider announcement). Log identifiers first: any change in reported model version or fingerprint is a drift event to investigate even if scores look fine.
6. Compare pass rates and scores against the baseline with an interval, not a single number: a two-proportion test or bootstrap interval on pass rate, and alert only when the change is outside the baseline's own run-to-run spread.
7. Look at per-category and per-check results, not only the average. Drift often shows up as one format or task class breaking while the overall mean barely moves.
8. When drift is confirmed, keep the evidence: baseline receipt, comparison receipt, the items that flipped and their outputs. That is what a provider support ticket or an internal incident review needs.

Silent drift is real and hard to see from inside one session

Chen, Zaharia and Zou (2023) measured GPT-3.5 and GPT-4 in March and June 2023 and found large behaviour changes on the same tasks under the same names, including a sharp drop in GPT-4's accuracy at identifying prime numbers and changes in how often generated code ran directly.

• Anthropic's September 2025 postmortem describes three overlapping infrastructure bugs (a routing error, output corruption and a compiler issue in token selection) that intermittently degraded Claude responses from early August to early September 2025, with no model change. Anthropic wrote that its own evaluations didn't capture the degradation and that it would run quality evaluations continuously on production systems. The same idea works from the customer side.
• Because the model ID didn't change in either case, a config audit would have shown nothing. Only replaying fixed inputs and comparing results with a recorded baseline shows it.

Why exact output matching alone gives false alarms

OpenAI describes seed as best effort and says determinism is not guaranteed. Thinking Machines' 2025 write-up on nondeterminism in LLM inference shows a main cause: kernels that aren't batch-invariant, so the same prompt at temperature 0 can produce different tokens depending on server load.

• So treat a changed output hash as a hint and use it to choose items to inspect. Use the baseline's repeat runs to learn the normal exact-match rate; a fall well below that rate, or flips concentrated in one task type, is more meaningful than any single mismatch.
• Deterministic checks are the most stable signal: valid JSON against a schema, required fields, refusal markers, length limits. They don't depend on wording, so a drop in their pass rate is hard to explain by sampling noise.

Keeping the baseline honest

A drift claim is only as good as the baseline. If the baseline lives in a spreadsheet you can edit, anyone (including you) can argue it was adjusted after the fact. Seal the baseline run before analysing it: hash the per-item results file and record it with a signed timestamp.

• Record each comparison run the same way, with the same canary-set hash and settings hash, so a reader can see that the two runs are comparable and which one came first.
• If your agent keeps its own hash-chained log of drift checks and commits the head to a ledger, you can later disclose just the relevant stretch (for example the week drift started) and others can confirm it wasn't edited, without publishing the whole log.