Answers · AI evidence
How do I investigate a customer's complaint that our AI chatbot made something up?
Last updated
TL;DR: Preserve first, then trace. Pull the conversation and your server's record of each model call behind it (the exact request with system prompt and retrieved documents, the model the provider reports, the settings, the exact output and the provider's request id) before changing anything, because asking the bot again won't reproduce the answer and providers keep their own copies for about 30 days. Compare the customer's copy with your logged output, then find where the claim came from: if it isn't anywhere in what the model was given, the model invented it; if it is, a document or tool result was wrong or out of date; if the system prompt told the bot to say it, it's a prompt problem; if your logged output differs from what the customer saw, something after the model changed it. Each answer leads to a different fix and a different reply to the customer. If you recorded a signed receipt when the reply went out, you can check the disputed text against it in seconds.
What should I check, step by step?
1. Get the customer's copy and the time: screenshot, transcript, email or chat export. Find the conversation in your logs by time and session or account id.
- Pull your server-side record of every model call in that conversation: request body, the model value in the response, settings, output, response id and the provider's request id (x-request-id on OpenAI; the request-id header, a req_ value, on Anthropic). OpenAI recommends logging request ids in production and lets you set your own X-Client-Request-Id.
- Move fast if you need the provider. OpenAI's data controls page lists abuse-monitoring logs kept up to 30 days by default, and Anthropic says it deletes API inputs and outputs within 30 days by default. Provider support may be able to look a request up by its id while it exists; don't count on it as your record.
- Compare the customer's text with your logged output exactly. If they differ, find the step: post-processing, templating, translation, a cache, the chat widget, or an edited copy.
- Search the retrieved passages and tool results in that request for the claim. Found: pull that document's version as of that day. Not found: the model generated it. Also read the system prompt for instructions that produced it, such as an instruction to always offer a refund.
- Don't use a re-run as proof either way. OpenAI says model outputs are variable by nature and seed is best effort, so a clean answer today doesn't show the bad one didn't happen, and a bad one today doesn't show it's what the customer got.
- Save the evidence before fixing: hash the request, output and source documents, and keep the prompt-template commit. Then fix, and keep the before and after configurations so you can show what changed.
- Reply to the customer based on what you found. Who bears the cost of a wrong answer is a separate question; the related guide on responsibility for a chatbot's wrong information covers Moffatt v. Air Canada.
Why doesn't re-asking the chatbot the same question show what it told the customer?
Re-running uses today's model, today's documents and today's prompt template. Any of them may have changed since the complaint, and even with nothing changed, the same request can give a different answer.
- The answer that matters is the one the customer got. Only a record kept when it was sent can show that, so the investigation starts from logs, not from the chat window.
How do I tell whether a chatbot invented a fact, repeated a bad source, followed its prompt or was changed afterwards?
Invented: the claim isn't in the system prompt, the conversation or any retrieved passage or tool result. This is a hallucination in the strict sense. Fixes are grounding, refusing when retrieval finds nothing, and checks on the output.
- Repeated: the claim is in a retrieved document or tool result that was wrong or stale. The model did what it was asked. The fix is the source and its update process, and the evidence is the document version on that day.
- Instructed: the system prompt or a few-shot example told the bot to say it, or to always give an answer. The fix is the prompt, and the prompt-template commit shows what was live at the time.
- Changed after the model: the logged output is right but the customer saw something else, from post-processing, translation, caching or display. The fix is in your pipeline, and the evidence is the gap between the logged output and the customer's copy.
What should I log for each chatbot reply so complaints about made-up answers are quick to investigate?
Per call: the exact request (with retrieved context and tool results), the provider's request id, the model value from the response, settings, the exact output and the template commit, plus a SHA-256 of each.
- Per reply sent: a signed receipt made when the reply went out, so the disputed text can be checked against it by anyone, without opening your logs.
- Keep the originals, not only hashes. A hash shows a match; it can't give back the text. Keep them as long as complaints and claims can come in, which is usually longer than any provider keeps its copy.
How do I ask Rightshold about this (free, no key)?
Add the MCP server https://www.rightshold.com/mcp to Claude, Cursor, ChatGPT, Grok or VS Code, then ask: “Fetch the signed AI run receipt recorded when this chatbot reply was sent, then check whether the exact text the customer says they saw matches the recorded output hash.”
{
"tool": "verify_ai_run",
"arguments": {
"id": "air_<20 hex characters from the receipt id>"
}
}- Free, no key. Calling with {id} returns the receipt and its signature. Call again with {receipt, signature, output: "<the exact reply text the customer quotes>"} to get output_matches, signature_valid and on_record. Add prompt to compare the request too.
- A match means the text is byte-for-byte what was recorded when the reply went out. A mismatch can be a real difference or a copy issue (whitespace, line breaks, a translated or retyped quote), so compare with your logged output before concluding anything.
- To have receipts to check, record each reply as it's sent with record_ai_run (API key): model, provider, prompt and output hashes, params such as request_id and template_commit, ran_at.
- In recorded mode the receipt shows what you recorded, not that the provider served that model, and no receipt shows the answer was true.
Rightshold doesn't connect to your chatbot, logs or model provider, and it doesn't decide whether an answer was a hallucination or who should pay for it. It checks text against receipts that were recorded. If a complaint may become a legal claim, keep the evidence as it is and talk to a lawyer before replying. Not legal advice.
What are the sources?
- OpenAI API reference: Overview (x-request-id, X-Client-Request-Id, model outputs are variable)
- OpenAI API reference: Create chat completion (seed is best effort)
- OpenAI: Data controls in the OpenAI platform (abuse monitoring retention)
- Anthropic: Errors (request-id header on every response)
- Anthropic Privacy Center: How long do you store my organization's data?
- NIST AI 600-1: Generative AI Profile (incident logging, version history and metadata)
- Verified AI Runs on Rightshold
Related
- Who is responsible when an AI chatbot gives a customer wrong information?
- How to prove what an AI model said at a specific time
- Does a passing eval prove what happened in one AI incident?
- Verified AI Runs
Not legal advice. JSON · Markdown · All answers
Get a free API key in one call
No card and no email needed. 100 free credits; the license and policy checks stay free without a key.
curl -s -X POST https://www.rightshold.com/v1/accounts/signup -H "Content-Type: application/json" -d '{"accept_terms":true}'
export RIGHTSHOLD_API_KEY=rs_... # the api_key from the reply (shown once)
curl -s https://www.rightshold.com/v1/licenses/check -H "Authorization: Bearer $RIGHTSHOLD_API_KEY" \
-H "Content-Type: application/json" -d '{"license":"MIT","intended_use":{"commercial_use":true}}'