Answers · AI evidence
How do I check that the file an AI agent delivered is byte-for-byte the output recorded for its run, and wasn't changed afterwards?
Last updated
TL;DR: Bind the output to the run with a hash taken at the moment the run finishes, then let the person who receives the output recompute it. At the end of the run, take SHA-256 of the exact bytes the agent produced (the file, or the text as UTF-8) and put that hash in a signed run record before anyone sends the output anywhere. The client then hashes the file they actually received with a standard tool (sha256sum on Linux, shasum -a 256 on macOS, Get-FileHash in PowerShell, which uses SHA-256 by default) and compares it with the hash in the record. Equal means the delivered file is the recorded output; any change, even a single byte, a line ending or a re-save, gives a different hash. Most false alarms come from transport, not tampering: line endings converted between LF and CRLF, Unicode normalization, JSON re-serialized with different key order or whitespace, or a PDF or image re-saved by an editor. Decide which bytes are the deliverable, hash those, and ship those bytes unchanged. This page is about checking a delivered output after the fact. Tying an output to the model version, prompt and settings that produced it is covered on a separate page.
What should I check, step by step?
1. Decide what the deliverable is before the run ends: the exact file(s), or the exact text. If you post-process the model's output (formatting, redaction, conversion to PDF), hash the version you will deliver, and record the raw output hash separately if you also want to show what the model returned.
- Hash the bytes, not a rendering. For a file, hash the file as written to disk. For text, hash its UTF-8 bytes exactly as delivered, including the final newline or its absence.
- Write the hash into a signed, timestamped run record at the end of the run, before the output leaves your system. A hash added after delivery proves only what existed when it was added.
- For more than one file, write a manifest: one line per file with its SHA-256 and path (the format sha256sum prints), sorted by path. Hash the manifest file and record that hash; the client checks the manifest's hash first, then each file with sha256sum -c.
- Ship the deliverable in a way that doesn't rewrite it: an archive or binary attachment, not pasted text. Mark exact-bytes files in Git as binary or -text so end-of-line conversion is off, and don't let an editor or office suite re-save them before sending.
- Give the client the run record id and the hash, plus one line on how to check it: sha256sum file, shasum -a 256 file or Get-FileHash file, then compare with output_sha256 in the record.
- If the client gets a mismatch, compare before you conclude tampering: check line endings and encoding first, and for JSON compare the parsed content. If a re-serialized format must match, agree on a canonical form (for JSON, RFC 8785) and hash that form at the end of the run.
- Keep your own copy of the exact delivered bytes. The record holds only the hash, so the original is what you hand over if the delivery is ever disputed.
What does binding an AI agent's output to a recorded run hash prove?
SHA-256 (NIST FIPS 180-4) maps any input to a 256-bit digest, and finding a different input with the same digest isn't practical. If the delivered file hashes to the output_sha256 in a run record, it is the same bytes that were recorded.
- The binding is only as strong as when it was made. A run record signed and written to a ledger at the end of the run fixes the hash at that time, so a later edit to the file can't be made to match without the change showing. A hash typed into an email or invoice later proves much less.
- Supply-chain attestation formats work the same way: an in-toto Statement names each artifact (subject) by its digest and treats subjects as immutable. A run record does the same for an agent's output.
Why can an unchanged AI output file fail a SHA-256 hash check?
Line endings. Git's text and eol attributes can convert LF to CRLF on checkout, and many tools do the same. The text reads the same and the bytes differ.
- Unicode normalization. The same visible character can be stored as one code point or as a base letter plus a combining mark. Unicode's normalization forms (UAX #15) treat those sequences as canonically equivalent, but the bytes, and so the hash, differ.
- Re-serialization. Parsing and re-writing JSON can change key order, spacing and number format. If the deliverable is structured data that will be re-serialized, hash a canonical form such as the JSON Canonicalization Scheme (RFC 8785), and tell the client to canonicalize before hashing.
- Re-saving. Opening a PDF, image or document and saving it again usually rewrites metadata and layout bytes, even with no visible change. Hash and send the file as written by the run.
How can a client check a delivered AI output against its run record?
With the receipt id, anyone can fetch the signed record for free with verify_ai_run and read record.output_sha256, then compare it with the hash of the file they received. For text outputs, verify_ai_run also accepts the output text and returns output_matches.
- Without Rightshold's servers: check the Ed25519 signature over the receipt's signed_bytes with the public key at /.well-known/rightshold-keys.json, and check that its ledger_seq is included in the ledger with the open verifier. ledger_verify (free) recomputes the hashes, chain and signatures of the ledger the receipt sits in.
- A match proves the file is the output that was recorded, and when. It doesn't prove the output is correct, or which model produced it: in recorded mode the caller reports the model.
How do I ask Rightshold about this (free, no key)?
Add the MCP server https://www.rightshold.com/mcp to Claude, Cursor, ChatGPT, Grok or VS Code, then ask: “Record a signed receipt for this agent run with the SHA-256 of the exact report file I am delivering to my client, so they can check later that the file they received is the recorded output.”
{
"tool": "record_ai_run",
"arguments": {
"model": "<model id the provider reported>",
"provider": "<provider>",
"prompt_sha256": "<sha256 of the exact request>",
"output_sha256": "<sha256sum of the delivered file>",
"params": {
"deliverable": "report.pdf",
"manifest": "none",
"delivered_via": "client portal"
},
"ran_at": "2026-11-07T22:00:00Z"
}
}- Needs an API key. Send output_sha256 for a file (the output of sha256sum), or the text itself as output, which is hashed on arrival and never stored. params is stored only as params_sha256, so keep the exact object.
- verify_ai_run is free and needs no key: send {id} to fetch the signed receipt and read record.output_sha256, or {receipt, signature, output} to check the signature, that it is on record, and that a text output matches.
- ledger_verify is free and needs no key: it recomputes every hash, chain link and signature on the Rightshold ledger the receipt is written to.
- For several files, record the SHA-256 of a sorted sha256sum manifest as output_sha256 and give the client the manifest. They check it with sha256sum -c.
Rightshold records the hashes you send, signs them and lets anyone check them later. It doesn't store your output, deliver it or check its content, and in recorded mode it can't attest which model produced it. A hash match shows the file is unchanged since it was recorded, not that it is correct. Not legal advice.
What are the sources?
- NIST FIPS 180-4: Secure Hash Standard (SHA-256)
- sha256sum(1) manual page (compute and --check SHA-256 digests)
- Microsoft Learn: Get-FileHash (SHA256 by default)
- Git documentation: gitattributes (text, eol and line-ending conversion)
- Unicode Standard Annex #15: Unicode Normalization Forms
- RFC 8785: JSON Canonicalization Scheme (JCS)
- in-toto Attestation Framework: Statement (subjects identified by digest)
- Verified AI Runs on Rightshold
Related
- How to prove which model version, prompt and settings produced an AI output
- How to prove what an AI model said at a specific time
- How to give a client verifiable evidence of which tools an AI agent called during a run
- Verified AI Runs
Not legal advice. JSON · Markdown · All answers
Get a free API key in one call
No card and no email needed. 100 free credits; the license and policy checks stay free without a key.
curl -s -X POST https://www.rightshold.com/v1/accounts/signup -H "Content-Type: application/json" -d '{"accept_terms":true}'
export RIGHTSHOLD_API_KEY=rs_... # the api_key from the reply (shown once)
curl -s https://www.rightshold.com/v1/licenses/check -H "Authorization: Bearer $RIGHTSHOLD_API_KEY" \
-H "Content-Type: application/json" -d '{"license":"MIT","intended_use":{"commercial_use":true}}'