Answers · agent safety
How do I give a browser-using AI agent per-origin and per-action permissions, and keep cookies and session storage out of the tools the model can see?
Last updated
TL;DR: Put the rules in the code that drives the browser, not in the prompt. Give each task its own browser context and a list of origins it may open (scheme + host + port, not 'anything on example.com'), and for each origin the actions it may take: read, fill a form, click submit, download, upload. Check every navigation, request and action against that list before it runs, and refuse anything off the list. Keep the session out of the model's reach: the tools the model calls should take a target and an action (open this URL, click this element, type into this field) and return page content, never cookies, localStorage, sessionStorage, auth headers or a saved storage-state file. Don't expose raw script evaluation, DevTools Protocol access or 'get cookies' as tools, because those can read session tokens, including HttpOnly cookies, that page scripts can't. Write each allow or deny decision to a record the agent can't edit, so you can show later what it was permitted to do and where.
What should I check, step by step?
1. Start one fresh browser context (or profile) per task, with no saved logins except the ones that task needs. Never point the agent at your everyday browser profile.
- Write the task's origin list: exact origins like https://app.example.com, not wildcards, and nothing reached only by redirect. Treat a new origin mid-task as a stop-and-ask, not an allow.
- For each origin list the allowed actions (read, fill, submit, download, upload, sign in, pay). Read-only research needs only read; a form task gets fill and submit on that one origin.
- Enforce it below the model: check the target URL before every navigation, filter network requests at the browser layer (for example Playwright's context.route), and block service workers so requests can't slip past the filter.
- Make the model-facing tools narrow: navigate, click, type, read text, take a screenshot. No tool returns cookies, storage, headers or the storage-state file, and none runs arbitrary JavaScript or DevTools Protocol commands.
- Keep saved session files (Playwright storageState and the like) on the host side, out of the agent's working folder, out of git, and out of logs and screenshots.
- Redact before anything goes back to the model: strip Authorization, Cookie and Set-Cookie headers, tokens in URLs, and hidden form fields from page content and network logs.
- Record every decision (task, origin, action, allow or deny, rule version) outside the agent, and clear the context's cookies and storage when the task ends.
Why can't a browser agent's permissions live in its prompt?
Pages are untrusted input. Anthropic's computer use docs warn that instructions on webpages or in images can override your instructions, and recommend a dedicated VM or container with minimal privileges, keeping account login information away from the model, and limiting internet access to an allowlist of domains.
- Google's Gemini Computer Use docs give the same advice for browsing agents: filter where the model can navigate and what it can do, with a blocklist as a starting point and a more restrictive allowlist being safer.
- If the only thing stopping the agent from opening a new site or pressing 'submit' is a line in its system prompt, one injected instruction on a page can undo it. A check in the code that runs the browser doesn't read the page.
What do per-origin, per-action permissions look like for a browser agent?
An origin is scheme, host and port together; it's the unit browsers use to separate one site's data from another's. Use it as the unit of permission too, so https://app.example.com doesn't quietly cover https://admin.example.com.
- Pair each origin with verbs. A price-comparison task might get read on three shop origins and nothing else. An expense task might get read, fill and submit on the expense app, and read only on the email app it pulls receipts from.
- Treat sign in, pay, upload, delete and change settings as separate verbs that a task doesn't get unless it was given them by name.
- Check each step, not just the start URL. Redirects, pop-ups, new tabs and links clicked mid-task are new navigations and go through the same check.
How do I keep cookies and session storage out of the model's view?
A saved browser state is a login. Playwright's authentication docs warn that the storage-state file may contain sensitive cookies and headers that could be used to impersonate you, and say not to check it into any repository. Its saved state covers cookies, localStorage, IndexedDB (when asked) and passkey-based login; sessionStorage needs extra code.
- HttpOnly stops page scripts from reading a cookie, but a tool with DevTools Protocol access can still read it. So 'the model can run any CDP command' or 'the model can evaluate any script' means 'the model can read the session'.
- Give the model results, not handles: the text of the page, a screenshot, whether a click worked. The browser driver holds the context and its cookies; the model never needs them to click a button.
- If a task needs a login, load the state into the context on the host side, or hand the sign-in step to a person (see the handoff guide), so the credential never passes through the model's context window.
What are the common failure modes?
The agent runs in the user's real browser profile, so it has every session the user has, on every site.
- The allowlist checks only the first URL, and a redirect or new tab takes the agent somewhere else that's already signed in.
- A generic 'execute JavaScript' tool is exposed for convenience, and a prompt injection uses it to read document.cookie or localStorage and send it out.
- The storage-state file sits in the agent's working folder, where a file-reading tool or a commit picks it up.
- Network logs and screenshots with tokens in them are fed back to the model or kept in plain logs.
How do I ask Rightshold about this (free, no key)?
Add the MCP server https://www.rightshold.com/mcp to Claude, Cursor, ChatGPT, Grok or VS Code, then ask: “Write down this browser agent's permissions for the task (the exact origins it may open and the actions allowed on each) as a declared intent, so sub-agents can only narrow it and anyone checking later can see what it was allowed.”
{
"tool": "declare_intent",
"arguments": {
"agent_id": "<one of your agents>",
"purpose": "other",
"description": "Expense task: read+fill+submit on https://expenses.example.com; read only on https://mail.example.com; no sign-in, payment, upload or other origins",
"operations": [
"record_ai_run"
],
"ttl_hours": 4,
"targets": {
"domains": [
"expenses.example.com",
"mail.example.com"
],
"resources": [
"action:expenses.example.com:read,fill,submit",
"action:mail.example.com:read"
]
}
}
}- Needs your account API key. Rightshold returns a canonical intent statement for the agent's key to sign (sign_intent). A child intent declared beneath it must keep its domains and resources within the parent's.
- This is a record, not a browser sandbox. Rightshold doesn't see or block your agent's navigation, clicks or cookies; the origin and action checks run in your browser driver as described above.
- record_ai_run can then write each allow or deny decision (origin, action, rule version) as a signed entry outside the agent.
Rightshold records declared intents and the decisions your browser driver sends it. It doesn't run, watch or control the browser, and it never needs or stores your cookies or session state. Not legal advice.
What are the sources?
- Claude Docs: Computer use tool (security considerations)
- Gemini API docs: Computer Use (best practices, allowlists and blocklists)
- Playwright docs: Authentication (storage state warning, session storage)
- MDN: Same-origin policy
Related
- How to sandbox an AI agent to one folder with a network allowlist
- How to hand off login and payment steps from a browser AI agent to a human with an audit record
- How to give an AI agent a scoped, expiring credential instead of a long-lived API key
- How to record an AI agent's trigger, permission decision and tool call outside its own memory
Not legal advice. JSON · Markdown · All answers
Get a free API key in one call
No card and no email needed. 100 free credits; the license and policy checks stay free without a key.
curl -s -X POST https://www.rightshold.com/v1/accounts/signup -H "Content-Type: application/json" -d '{"accept_terms":true}'
export RIGHTSHOLD_API_KEY=rs_... # the api_key from the reply (shown once)
curl -s https://www.rightshold.com/v1/licenses/check -H "Authorization: Bearer $RIGHTSHOLD_API_KEY" \
-H "Content-Type: application/json" -d '{"license":"MIT","intended_use":{"commercial_use":true}}'