How do I make an AI agent's side-effecting tool calls (payments, emails, tickets, deploys) idempotent, cap retries per task, and require a new approval when the amount, recipient or scope changes?

What should I check, step by step?

1. List the agent's tools that change something outside the run: payments, refunds, emails and messages, tickets, calendar invites, file writes, deploys, permission grants. Every one of these needs an action key; read-only tools don't.

2. Create the key once, when the agent commits to the action, from the task id plus a step id (or a random UUID stored with the step). Never generate it inside the function that sends the request, or each retry looks like a new action.
3. Pass the key to the API if it accepts one (Stripe's Idempotency-Key header, x402 payment-identifier, provider client-request ids). Otherwise write the key and the outcome to your own store before and after the call, and check the store before any retry.
4. Define the action hash: a SHA-256 of the canonical JSON (for example RFC 8785) of tool name, arguments, amount in minor units, currency, recipient or payee, and requested scope. Keep it next to the key.
5. Set retry caps in the tool wrapper: for example 3 attempts per action and 10 retries per task, with exponential backoff. Override framework defaults that retry without limit.
6. Mark errors that won't fix themselves (422 validation, 401/403, policy deny, insufficient funds, recipient not found) as non-retryable, and send them to a person or back to the planner with the reason.
7. Before retrying with a changed action, recompute the hash. If it differs from the approved hash, cancel the old approval and request a new one; never edit the amount or recipient under an existing approval.
8. After a timeout, reconcile before retrying: look up the charge, the sent message or the created ticket by its key. An unknown result is not a failure.

Why do AI agents repeat side effects like payments and emails?

Retries come from several layers at once: the HTTP client, the agent framework, the durable-execution engine, and the model itself calling the same tool again when the output was unclear. Each layer can resend a payment or an email unless the action carries a stable key.

• Temporal's docs say activities get a default retry policy with exponential backoff (coefficient 2) and unlimited attempts, while workflow executions don't retry by default. An activity that charges a card or sends a message has to be idempotent or carry an explicit cap.
• LangGraph's interrupt docs warn that code before interrupt() runs again when the graph resumes, so side effects placed there must be idempotent or moved after the interrupt.
• AWS's Builders' Library describes the same fix for service APIs: the caller supplies a unique request token, and the service returns the original result for a repeat with the same token instead of doing the work twice.

How do idempotency keys and action hashes work together?

The idempotency key answers 'is this the same attempt?'. The action hash answers 'is this the same thing a person approved?'. Keep both: the key stops duplicates, the hash stops drift.

• The IETF Idempotency-Key draft says a key must not be reused with a different payload, and suggests 422 when that happens and 409 while the first request is still running. Stripe returns an error if parameters change under a key it has seen. Your own ledger should do the same.
• Put everything that matters to the approver in the hash: amount and currency, payee or recipient list, the target account or repo, the permission scope requested, and the tool name. Leave out fields that legitimately vary per attempt, such as a timestamp header.
• Store the approval with its hash and an expiry. A retry within the expiry and with the same hash can proceed; anything else needs a new decision.

How do I set per-task retry caps an agent can't raise?

Count attempts in the tool wrapper or gateway, keyed by task id and action key, not in the prompt. A cap the model can read and argue with is a suggestion.

• Use two numbers: attempts per action (to stop one call hammering a failing API) and total retries per task (to stop a confused agent working through many similar actions).
• When a cap is hit, stop the action, record why, and escalate. Don't let the model rephrase the action to get a fresh counter; the new action gets a new hash and, for anything that needed approval, a new approval.
• Backoff matters less than the cap. Exponential backoff spreads retries out; it doesn't stop a payment going out twice.

When must an agent's retry go back to a human for approval?

Amount went up (or the currency changed), even slightly: new hash, new approval.

• Recipient, payee, mailing list or ticket assignee changed: new approval. This is the common prompt-injection path, where a retried action quietly points somewhere else.
• Scope widened: write instead of read, a new repo, a broader OAuth scope, a production target instead of staging. New approval.
• The approval expired, or the task it was granted for ended. New approval, even if the hash matches.

What are the common failure modes?

The key is generated inside the retry loop, so three timeouts send three emails.

• An approval is stored as approved: true with no hash, and the retried call pays a different merchant.
• The framework's default retry policy retries an insufficient-funds error for hours.
• One key is reused for two different invoices; the second silently returns the first one's result.
• A timeout is treated as failure and retried, but the first call succeeded; reconcile by key first.