Rightshold™ Sign up

Answers · AI evidence

How do I give a client verifiable evidence of which tools an AI agent called during a run, without handing over the arguments, results or secrets?

Last updated

TL;DR: Give the client a tool-call manifest, not your logs. Build it from the record your tool layer writes (not from the model's own account): one line per call, in order, with the sequence number, tool name, server, call id, start time, outcome, and a salted SHA-256 of the canonical arguments and of the result. Hash the whole manifest and seal that one hash in a signed, timestamped run record at the end of the run. The client gets the manifest and the record id. They can see which tools ran, in what order and whether each succeeded, recompute the manifest hash and confirm it matches the sealed record, so no line was added, dropped or reordered afterwards. Arguments and results stay with you. If the client later needs to see one call, you hand over that call's arguments and salt, and they check it against the hash already in the manifest. The salt matters: without it, short or predictable arguments can be guessed by hashing candidates. This page is about client-facing evidence. Capturing tool calls at a gateway and keeping an internal audit log are covered on separate pages.

What should I check, step by step?

1. Take the calls from the layer that actually executes tools (your MCP client, gateway or tool runner), not from the model's narration. The model can describe a call it never made or leave one out.

  1. For each call record: seq (1, 2, 3 in execution order), tool name, server or provider, call id, started_at, outcome (ok, error, denied), and whether a human approved it. OpenTelemetry's execute_tool span convention requires gen_ai.tool.name and uses gen_ai.tool.call.id, with arguments and results as opt-in content, which is a good model for what's safe to show.
  2. For arguments and results, store a salted hash instead of the content: generate a fresh random salt per call (at least 128 bits), then SHA-256 over the salt plus the canonical JSON of the arguments (RFC 8785). Do the same for the result. Keep the salts and the originals privately.
  3. Write the manifest as one canonical JSON line per call, sorted by seq, and add a header with the run id, the tool list the agent was allowed to use and the count of calls. Take SHA-256 of the whole manifest file.
  4. At the end of the run, seal the manifest hash and call count in a signed run record (with record_ai_run, in params). Do it before you send the client anything, so the record's time comes before the delivery.
  5. Send the client the manifest file and the receipt id. They check that the manifest hashes to the value in the record, then read the tool names, order and outcomes directly.
  6. When the client asks about a specific call, send that call's arguments (or result) and its salt. They recompute the salted hash and compare it with that line of the manifest. Nothing else in the run is revealed.
  7. Never put tokens, API keys or personal data in the manifest in clear text. If a value matters to the client, it goes in through the salted hash and is disclosed only if needed.

Why isn't an AI agent's own transcript evidence of which tools it called?

A transcript or chat log is written by the same system whose behaviour is in question, and it's easy to edit. A tool-call record written by the code that runs the tools, and sealed by someone else at the end of the run, is a much stronger basis for what the client is told.

How do salted hashes let me reveal one AI agent tool call without the rest?

A plain hash of a short argument (a customer id, a yes/no flag, a small amount) can be reversed by hashing every likely value. SD-JWT (RFC 9901) solves the same problem for identity claims by hashing each disclosable value together with a random salt and sending the salt only when the value is disclosed. The manifest uses the same idea per call.

What can a client verify from a sealed AI agent tool-call manifest?

Can verify: the manifest they hold is the one sealed at the end of the run (verify_ai_run on the receipt, then recompute params_sha256 from the params object you give them and the manifest hash inside it); the record's signature and ledger entry (offline with the published key, or ledger_verify); and any call you choose to open.

How do I ask Rightshold about this (free, no key)?

Add the MCP server https://www.rightshold.com/mcp to Claude, Cursor, ChatGPT, Grok or VS Code, then ask: “Record a signed receipt for this agent run that seals the SHA-256 of the tool-call manifest I'm sending my client and the number of calls, without uploading the calls themselves.”

One-click setup

{
  "tool": "record_ai_run",
  "arguments": {
    "model": "<model id the provider reported>",
    "provider": "<provider>",
    "prompt_sha256": "<sha256 of the exact request>",
    "output_sha256": "<sha256 of the delivered output>",
    "params": {
      "tool_call_manifest_sha256": "<sha256 of the manifest file>",
      "tool_call_count": 14,
      "manifest_format": "jsonl, one line per call, sorted by seq, salted arg/result hashes"
    },
    "ran_at": "2026-11-07T22:00:00Z"
  }
}

Rightshold signs and timestamps the hashes and fields you send and lets anyone check them later. It doesn't run your tools, capture your tool calls or see the manifest, so it can't confirm the manifest is complete. Not legal advice.

What are the sources?

Related

Not legal advice. JSON · Markdown · All answers

Get a free API key in one call

No card and no email needed. 100 free credits; the license and policy checks stay free without a key.

curl -s -X POST https://www.rightshold.com/v1/accounts/signup -H "Content-Type: application/json" -d '{"accept_terms":true}'
export RIGHTSHOLD_API_KEY=rs_...   # the api_key from the reply (shown once)
curl -s https://www.rightshold.com/v1/licenses/check -H "Authorization: Bearer $RIGHTSHOLD_API_KEY" \
  -H "Content-Type: application/json" -d '{"license":"MIT","intended_use":{"commercial_use":true}}'