Answers · agent safety
How do I bind an agent approval to a specific tested artifact or version?
Last updated
TL;DR: Treat release approval as change control, not a free-floating yes. The human decision must name the exact artifact that was tested — image digest, firmware SHA-256, package version plus content hash, or commit that built it — and the deploy step must refuse anything else. Record the approver, the approved identity (digest/version), the test evidence pointer, and the released identity in a change record the agent cannot edit or overwrite. Frameworks pause for approval (OpenAI Agents SDK interruptions; GitHub Environments required reviewers) but do not by themselves prove the bytes on the plant or cluster match what was tested; your gate has to compare digests at load time. Pair request_human_approval (action hashed with artifact fields) with record_ai_run when you need a signed receipt of the tested build and the decision.
What should I check, step by step?
1. Identify the unit under change with an immutable fingerprint: container image sha256 digest, firmware or package SHA-256, signed build attestation, or the exact commit + reproducible build id — never a floating tag like latest or a branch name alone.
- Collect test evidence against that same fingerprint (CI run URL, signed test pack, holdout eval receipt). Reject evidence that names a different digest or an unpinned tag.
- Open a human approval whose action object includes artifact_id, version, content_digest (or image digest), environment, and evidence_ref. Hash those fields; do not approve a prose summary alone.
- Store the decision (approver, decided_at, approved digest/version, evidence_ref) in a server DB, MES/change system, or ledger the agent's tools cannot UPDATE. Give the agent only the approval or change-record id.
- At load/deploy time, re-read the live artifact digest and compare to the approved one. Fail closed on mismatch, missing approval, or expired grant — do not ship "close enough" builds.
- Append an immutable change record: who approved, which digest was approved, which digest was released, when, and the evidence pointer. Prefer append-only versions over edits to past rows.
- Re-check at execution (or SOFTWARE_LOAD / apply) time, not only when the ticket was filed: a yes for build A is not a warrant for build A' rebuilt after the yes.
- Keep only the change-record id in the agent session — never approved=true or a mutable copy of the digest the model can rewrite.
Why version labels alone are not enough
A human "approved v2.4" while CI rebuilt v2.4 from a different commit leaves production running untested bytes. Bind the yes to the digest (or equivalent fingerprint), and treat the human-readable version as a label on that digest.
- Kosli's release/promotion pattern records each Artifact's fingerprint (for example a Docker image sha256) and only promotes Artifacts that meet SDLC gates for the target environment; the human go/no-go sees exact incoming and outgoing identities, not just names.
- GitHub Environments can require reviewers before a job that uses that environment runs, and can limit which branches or tags may deploy — useful as a people-gate, still paired with a digest check in the deploy step itself.
- Industrial change practice expects the load step to accept only the exact expected version plus artifact hash; drift between the approved baseline and the live device is a separate compare job, not a chat memory.
Approval, then an uneditable change record
request_human_approval hashes the action object. Put artifact_id, version, content_digest, environment, and evidence_ref inside action so verify_human_approval fails if the agent swaps in another build after the yes.
- OpenAI's Agents SDK pauses on needsApproval and resumes from RunState; keep that snapshot on the server. Browser clients should see opaque ids and display fields, not a client-uploaded "already approved" flag.
- record_ai_run can store a signed receipt of the evaluation or test summary (model/provider if an AI scored it, or hashes of the test log and artifact digest) that verify_ai_run can check later — proof of what was recorded, not that the plant is safe.
- After release, write approver + approved digest + released digest into append-only storage. If someone must correct a mistake, add a new versioned row that supersedes the old one; do not silently rewrite history.
What can go wrong?
Approval UI shows the agent's summary ("roll out the tested build") while the tool arguments point at a different tag or digest.
- Floating tags (latest, release) move between approval and deploy; the digest gate was never wired.
- The only copy of approved_digest lives in the agent's session JSON and gets overwritten on the next turn.
- Change tickets record "approved by Alice" without the digest, so an auditor cannot tell which bytes left the warehouse.
How do I ask Rightshold about this (free, no key)?
Add the MCP server https://www.rightshold.com/mcp to Claude, Cursor, ChatGPT, Grok or VS Code, then ask: “Before releasing this build, ask me to approve the exact tested artifact (id, version, content digest, environment and evidence ref), bind the yes to a hash of those fields, and wait for my decision.”
{
"tool": "request_human_approval",
"arguments": {
"summary": "Release plant-controller build 2.4.1 (sha256:9f2c…) to line-3 after soak test SOAK-441; reject any other digest",
"action": {
"type": "release_artifact",
"artifact_id": "plant-controller",
"version": "2.4.1",
"content_digest": "sha256:9f2c…",
"environment": "line-3",
"evidence_ref": "ci://runs/SOAK-441",
"policy_version": 7
},
"expires_in_minutes": 120
}
}- Needs your API key. The action is hashed; you decide once with decide_human_approval. verify_human_approval fails if the digest, version or environment no longer match.
- record_ai_run stores a signed receipt of the test or eval run (hashes of logs and of the artifact digest) outside the agent session; verify_ai_run is free.
- How to re-check agent authority at execution time covers binding grants to policy_version on every risky call; this page is about binding the yes to the bytes that were tested.
- How to show a human the actual change before approving covers rendering diffs and tool arguments; pair that UI with the digest fields above.
Rightshold records human approvals and signed AI-run receipts for agents that call it. It does not flash firmware, push images, sit in your MES, or stop a deploy pipeline that never asks. Enforce the digest compare in your load/deploy gate too. Not legal advice.
What are the sources?
- Kosli: release/promotion workflow with exact Artifact fingerprints
- GitHub Docs: Managing environments for deployment (required reviewers; deployment branches/tags)
- OpenAI Agents SDK (JS): Human-in-the-loop (keep approval state on the server)
- Moduvio Docs: Signed software/test execution packs (exact version + artifact SHA-256 at SOFTWARE_LOAD)
- Human approvals on Rightshold
- Verified AI Runs on Rightshold
Related
- How to re-check agent authority at execution time
- How to show a human the actual change before approving an AI agent action
- How to keep an agent risk/approval record outside the agent's editable session
- How to audit which human authorized an AI agent action
Not legal advice. JSON · Markdown · All answers
Get a free API key in one call
No card and no email needed. 100 free credits; the license and policy checks stay free without a key.
curl -s -X POST https://www.rightshold.com/v1/accounts/signup -H "Content-Type: application/json" -d '{"accept_terms":true}'
export RIGHTSHOLD_API_KEY=rs_... # the api_key from the reply (shown once)
curl -s https://www.rightshold.com/v1/licenses/check -H "Authorization: Bearer $RIGHTSHOLD_API_KEY" \
-H "Content-Type: application/json" -d '{"license":"MIT","intended_use":{"commercial_use":true}}'
