Rightshold™ Sign up

Answers · AI evidence

How do I prove which model version, prompt and settings produced a specific AI output?

Last updated

TL;DR: Build a content-addressed run record at the moment the response comes back: the model ID the provider says actually answered (not the alias you asked for), a SHA-256 of the exact prompt bytes (system prompt, messages, tool schemas), a SHA-256 of the canonical settings object (temperature, top_p, max tokens, seed, tool_choice, response format), a SHA-256 of the exact output, and the provider's response id and backend fingerprint where one is returned. Because each part is named by its hash, anyone holding the original prompt, settings and output can recompute the hashes and see whether they match; change one byte and the match fails. Then get that record signed and timestamped by someone other than you, so it can't be quietly rebuilt later. This proves which configuration was recorded with which output, and when. It does not prove the provider really served that model, and it can't make the run repeatable. record_ai_run signs such a record onto Rightshold's ledger; verify_ai_run lets anyone check it for free.

What should I check, step by step?

1. Record the model the provider reports in the response, not only the one in your request. OpenAI's chat completion object returns model and, where available, system_fingerprint; Anthropic's Messages response returns model; Gemini's GenerateContentResponse returns modelVersion and responseId. If you requested an alias (for example a -latest name), the reported value is what pins the version.

  1. Hash the prompt as sent, not as templated in your code: serialize the full request messages (system prompt, user turns, tool results already in context) plus tool definitions to one byte string and take SHA-256. Keep the bytes; the hash is only useful if you can produce the original.
  2. Put every generation setting into one settings object (model requested, temperature, top_p, max tokens, seed, stop sequences, tool_choice, response_format or JSON schema, reasoning or thinking budget) and hash its canonical form with sorted keys, so key order can't change the hash.
  3. Hash the exact output bytes you returned to the user or the next step, after any post-processing you apply. If you post-process, record the raw output hash too and say which one is which.
  4. Keep the provider's request or response id and created time next to the hashes. They let you or the provider correlate the run later while logs still exist.
  5. Record the code version that built the prompt (git commit of the template and any retrieval config). A prompt hash without the template version tells you that something changed, not what.
  6. Seal the record once, at return time: sign it and get an independent timestamp, so a record rebuilt after a dispute is distinguishable from one made when the answer came back.
  7. Write down the limits next to the record: a caller-reported record can't prove which weights served the request, and seeds are best effort, so a re-run may legitimately differ.

Why 'we used model X' isn't enough

Aliases move. Anthropic's models overview says that for models before the 4.6 generation the short alias points to a dated snapshot ID (from 4.6 on, every ID is a pinned snapshot), and Gemini documents -latest style names that are updated over time. A config file that names an alias doesn't say which version answered on a given day; the model value in the response does.

What content-addressed means here, and how someone checks it

Each part of the run is identified by the SHA-256 of its exact bytes: prompt_sha256, output_sha256 and a hash of the canonical settings object. The record then carries those hashes plus the model and provider names, and the record itself is hashed and signed.

What this proves and what it doesn't

Proves: this exact output was recorded together with this prompt, these settings and this reported model, no later than the signed issued time, and none of it has changed since.

How do I ask Rightshold about this (free, no key)?

Add the MCP server https://www.rightshold.com/mcp to Claude, Cursor, ChatGPT, Grok or VS Code, then ask: “Record a signed run receipt tying this output to the reported model version, the hash of the exact prompt bytes and my generation settings.”

One-click setup

{
  "tool": "record_ai_run",
  "arguments": {
    "model": "claude-sonnet-4-5-20250929",
    "provider": "anthropic",
    "prompt_sha256": "<sha256 of the exact request messages + tool schemas>",
    "output_sha256": "<sha256 of the exact output bytes>",
    "params": {
      "requested_model": "claude-sonnet-4-5",
      "temperature": 0,
      "max_tokens": 1024,
      "tool_choice": "auto",
      "template_commit": "<git sha>",
      "response_id": "<provider response id>"
    },
    "ran_at": "2026-11-07T17:00:00Z"
  }
}

Rightshold records the hashes and fields you send, signs them and lets anyone check them later. It doesn't call your model in recorded mode, can't attest which weights a provider served, and doesn't judge whether an output is correct. Not legal advice.

What are the sources?

Related

Not legal advice. JSON · Markdown · All answers

Get a free API key in one call

No card and no email needed. 100 free credits; the license and policy checks stay free without a key.

curl -s -X POST https://www.rightshold.com/v1/accounts/signup -H "Content-Type: application/json" -d '{"accept_terms":true}'
export RIGHTSHOLD_API_KEY=rs_...   # the api_key from the reply (shown once)
curl -s https://www.rightshold.com/v1/licenses/check -H "Authorization: Bearer $RIGHTSHOLD_API_KEY" \
  -H "Content-Type: application/json" -d '{"license":"MIT","intended_use":{"commercial_use":true}}'