Answers · agent safety
How do I hand login and payment steps from a browser-using AI agent to a human, and keep an auditable record of each handoff?
Last updated
TL;DR: Make the handoff a step the browser driver enforces, not something the model decides to do. When the agent reaches a sign-in form, a passkey or one-time-code prompt, a CAPTCHA, a card or bank form, or a final 'place order' / 'pay' button, the driver pauses the agent, stops sending it screenshots, and gives the live page to a person. The person types the password or card details and confirms the payment themselves; the agent never sees them. When the person hands back, the driver records what happened: which task and origin, what kind of step (login, payment, CAPTCHA), who took over, when control passed each way, and for payments the exact merchant, amount and currency the person approved, bound to a hash of that action. If anything changes after approval (a different amount, payee or cart), the agent has to ask again. ChatGPT agent works this way: you take over its browser to sign in, it asks before purchases, and OpenAI says it doesn't collect what you type during takeover.
What should I check, step by step?
1. List the handoff steps for your agent: password and username forms, passkey and one-time-code prompts, CAPTCHAs and other human checks, card, bank and wallet forms, final purchase or pay buttons, and accepting terms or cookie banners.
- Detect them in the driver, not only by asking the model: password inputs, autocomplete=cc-number and similar fields, known payment iframes, WebAuthn prompts, and a list of URLs per origin. When one appears, pause the agent before it acts.
- During the handoff stop feeding the model screenshots, page text and keystrokes, and don't log the person's input. Resume only when the person says they're done.
- For payments, show the person the exact merchant, amount, currency and items before they confirm, and hash that action. Approval for one cart or amount doesn't cover another.
- Record each handoff as two entries, out and back: task id, origin, step type, reason, person, times, outcome (signed in, paid, cancelled, timed out) and, for payments, the action hash and approval id. Never the password, code or card number.
- Give handoffs a timeout and a safe default: if nobody answers, the task stops; it doesn't retry the login itself or pick another payment method.
- Re-ask on change: if the agent later needs a new origin, a higher amount, a different payee or a new login, that's a new handoff, not part of the old one.
- Review the record: count handoffs per task and flag payments with no matching approval, and logins on origins the task wasn't given.
Why should login and payment steps go to a person?
Credentials and payments are where a mistake or a prompt injection costs the most. Anthropic's computer use docs advise keeping account login information away from the model and asking a human to confirm actions with real-world consequences or that need affirmative consent, such as accepting cookies, completing financial transactions or agreeing to terms of service.
- Google's Gemini Computer Use model can return a safety_decision of require_confirmation; its docs say to prompt the end user and pass back a safety acknowledgement only if they confirm. Its sample safety instructions tell the model never to solve or bypass CAPTCHAs, and Google's API terms say you won't automatically bypass requests for human confirmation.
- Some steps can't be done by an agent anyway: a passkey asks the person at the device to confirm with a fingerprint, face or PIN, so the natural design is to hand the page to them.
How does ChatGPT agent hand login and payment steps to the user?
OpenAI's ChatGPT agent prompts you to log in by taking over its browser, asks for permission before actions with real-world consequences like making a purchase, needs active supervision ('watch mode') for tasks like sending email, and refuses high-risk tasks such as bank transfers.
- OpenAI says that in takeover mode your inputs stay private: ChatGPT doesn't collect or store data you enter, such as passwords, because the model doesn't need it and it's safer if it never sees it.
- You can copy that shape for your own agent: the person gets the live page, the model gets nothing from that stretch, and control comes back only when the person hands it back.
What should a human handoff record prove?
That the step was done by a person, not the agent: who took over, when, and that the agent was paused for that stretch.
- What exactly was approved: for a payment, the merchant origin, amount, currency and item list the person saw, as a hash, so a later dispute can check the charge against it.
- That the agent stayed in scope afterwards: the origin and task the session was opened for, so a login handed over for one site isn't used for another.
- Keep it outside the agent's memory and working folder, append-only, with the approval and the outcome linked by id.
What are the common failure modes?
The model is given the password 'just for this site', and it ends up in the prompt history, logs or a screenshot.
- Screenshots keep streaming to the model while the person types their card number.
- The approval says 'buy the headphones' and the agent later adds a warranty or changes the shipping speed on the same yes.
- The handoff happens but nothing is written down, so after a disputed charge nobody can show who confirmed it or what they saw.
- A CAPTCHA is treated as a bug for the agent to work around instead of a stop for a person.
How do I ask Rightshold about this (free, no key)?
Add the MCP server https://www.rightshold.com/mcp to Claude, Cursor, ChatGPT, Grok or VS Code, then ask: “My browser agent has reached the payment step. Ask me to approve the exact purchase (merchant origin, amount, currency and items), and I'll complete the payment myself in the browser; keep the approval as a signed record.”
{
"tool": "request_human_approval",
"arguments": {
"summary": "Browser agent handoff at checkout on shop.example.com: 1 pair of headphones, 89.00 USD. I'll enter payment details myself.",
"action": {
"type": "browser_handoff",
"step": "payment",
"origin": "https://shop.example.com",
"amount_minor": 8900,
"currency": "USD",
"items": [
"headphones-x1"
],
"task_id": "task_…"
},
"expires_in_minutes": 15
}
}- Needs your API key. The action is hashed, and you decide once with decide_human_approval (approve or deny); the decision is a signed, timestamped receipt. Card numbers and passwords never go to Rightshold.
- verify_human_approval (free) checks a decision receipt and that the action matches what was approved, so a changed amount, payee or cart doesn't pass on the same yes.
- record_ai_run can write the handoff-out and handoff-back entries (step type, origin, times, outcome, approval id) as signed records outside the agent.
Rightshold records approvals and handoff entries your driver sends it. It doesn't run or watch the browser, take over sessions, see what anyone types, or process the payment. Not legal or payment-compliance advice.
What are the sources?
- OpenAI: Introducing ChatGPT agent (takeover mode, confirmations, watch mode)
- Claude Docs: Computer use tool (security considerations)
- Gemini API docs: Computer Use (safety decisions and acknowledgement)
- W3C: Web Authentication Level 2 (user presence and verification)
Related
- How to give a browser AI agent per-origin permissions and keep cookies out of its tools
- How to require human approval before an AI agent spends money
- How to show a human the actual change before approving an AI agent action
- How to audit which human authorized an AI agent action
Not legal advice. JSON · Markdown · All answers
Get a free API key in one call
No card and no email needed. 100 free credits; the license and policy checks stay free without a key.
curl -s -X POST https://www.rightshold.com/v1/accounts/signup -H "Content-Type: application/json" -d '{"accept_terms":true}'
export RIGHTSHOLD_API_KEY=rs_... # the api_key from the reply (shown once)
curl -s https://www.rightshold.com/v1/licenses/check -H "Authorization: Bearer $RIGHTSHOLD_API_KEY" \
-H "Content-Type: application/json" -d '{"license":"MIT","intended_use":{"commercial_use":true}}'