{
  "question": "How do I return tiered API responses for verified AI agents vs anonymous bots, without blocking human browsers?",
  "slug": "how-to-return-tiered-api-responses-for-verified-ai-agents-vs-anonymous-bots",
  "url": "https://www.rightshold.com/answers/how-to-return-tiered-api-responses-for-verified-ai-agents-vs-anonymous-bots",
  "short_answer": "Serve capability tiers, not a single allow/deny wall. Classify every request into a response class before you build the body: human browser (session or ordinary HTML), verified agent (short-lived credential or checked Web Bot Auth signature with scope), anonymous automation (no usable identity), and verified search crawler (allowlisted). Humans keep the full public page. Verified agents get a machine-shaped payload sized to their scope — for example catalog summaries at scope marketplace:browse, contracted prices and hold ids only at marketplace:paid_read. Anonymous bots get a stable public stub or 401 with WWW-Authenticate telling them how to present a credential (RFC 7235), not a CAPTCHA that burns your human challenge budget. This is graceful degradation by identity: the same URL can return different fields by Accept and by verified tier, without cloaking the human HTML search engines need. It is not a rate-limit page (that is about counting) and not a hard gate that refuses every unpaid field (that is covered separately).",
  "checklist": [
    "Define three or four response classes up front: public_summary, human_html, agent_scoped (keyed by credential scope), and deny_or_challenge for unverified automation on sensitive routes.",
    "Classify before serialization: verify the agent credential or Web Bot Auth signature first, then pick the response builder. Do not build the paid body and strip fields afterwards — that leaks timing and accidental fields.",
    "Keep human HTML honest and indexable on the marketing or product URL. Put dense machine feeds on /api/... or negotiate with Accept: application/json so crawlers and shoppers are not served a hollow page.",
    "For verified agents, map scope → fields. A browse-tier credential returns ids, titles and public prices; a paid-read credential adds contract prices, inventory holds and PII your agreement allows. Missing scope on an otherwise valid credential is 403 Forbidden (RFC 7235 / RFC 7231), not a silent empty list.",
    "For anonymous automation on agent-oriented routes, return 401 Unauthorized with a WWW-Authenticate challenge that names your agent credential scheme and a link to registration docs. Do not send that client through a browser CAPTCHA.",
    "Use one stable JSON error shape (code, message, how_to_upgrade) so agents can branch. Prefer not to reveal whether a hidden SKU exists when the caller lacks the tier that may see it.",
    "Log class, credential id (if any), scope and HTTP status so you can see whether humans were accidentally folded into the bot path.",
    "Test three clients on the same path: a logged-out browser, a credentialed agent, and curl with no auth. The browser must still get usable HTML; only the agent path should change with scope."
  ],
  "sections": [
    {
      "heading": "Tiers are response shapes, not just allow or deny",
      "points": [
        "A hard gate answers yes or no. A tiered response answers how much: public stub, scoped agent JSON, or full human page. Marketplaces and SaaS APIs need the middle option so cooperative agents can work without you opening every field to scrapers.",
        "RFC 7235 says a 401 must carry WWW-Authenticate with at least one challenge when the request lacks valid credentials; a server that has valid credentials that still are not enough for the resource should use 403. Use that split: unknown caller → 401 + how to get a credential; known agent lacking scope → 403 with a machine-readable upgrade hint.",
        "Cloudflare's bot products separate verified bots, likely automated traffic and humans. Mirror that idea at your origin for response content: verification changes which serializer runs, not only whether a WAF rule fires."
      ]
    },
    {
      "heading": "Classify the caller before you build the body",
      "points": [
        "Order of checks: (1) is this a verified search crawler you allowlist for HTML? (2) is there a valid, unrevoked agent credential (or Web Bot Auth signature) with scope for this route? (3) is there a human session? (4) otherwise treat as anonymous automation.",
        "verify_agent_credential (free) answers validity, revocation and optional scope in one call. Cache positive results briefly; fail closed on paid tiers if verification is down.",
        "Never key tiers only on User-Agent strings. Spoofable labels belong in logs, not in the branch that unlocks contract prices."
      ]
    },
    {
      "heading": "Keep humans on the full HTML path",
      "points": [
        "Graceful degradation fails when a bot fight breaks the shop. Do not put ordinary browsers into the anonymous-automation 401 path. Session or first-party cookie traffic stays on human_html even when an agent from the same NAT is limited.",
        "If you negotiate content types, document it: text/html for people and crawlers, application/json for agents with credentials. Same resource family, different representations — not a secret URL that only agents know while Google sees a blank page.",
        "Pair tiers with per-agent quotas on a separate control (covered on the rate-limit page). Tiers decide the body; quotas decide how often an agent may ask."
      ]
    }
  ],
  "checks": [],
  "ask_rightshold": {
    "start": "https://www.rightshold.com/start?ref=seo",
    "mcp_endpoint": "https://www.rightshold.com/mcp",
    "ask": "Before I build this API response, verify the caller's agent credential and tell me whether it still has marketplace:browse so I can return the browse-tier JSON instead of a 401.",
    "mcp_call": {
      "tool": "verify_agent_credential",
      "arguments": {
        "id": "crd_<id from the Authorization header>",
        "scope": "marketplace:browse"
      }
    },
    "also": [
      "Free and keyless over MCP or GET /v1/credentials/verify. On success, serialize the browse-tier body; on failure, return 401 with WWW-Authenticate and a how_to_upgrade link, or 403 if the credential is valid but lacks scope.",
      "issue_agent_credential (needs your API key) mints short-lived scoped credentials for each tier you sell (browse, paid_read, settle).",
      "declare_intent / precall_check can record the intended fetch and purpose before your handler runs so the audit trail shows why a tier was granted.",
      "revoke_agent_credential drops one abusive agent without changing human HTML or other agents' tiers."
    ],
    "scope": "Rightshold verifies credentials and records intents you send. It does not host your API or choose your field lists; your handler must pick the response class after verification. Not legal advice."
  },
  "sources": [
    {
      "title": "RFC 7235: HTTP Authentication (401, WWW-Authenticate; 403 when credentials are inadequate)",
      "url": "https://www.rfc-editor.org/rfc/rfc7235"
    },
    {
      "title": "RFC 7231: 403 Forbidden",
      "url": "https://www.rfc-editor.org/rfc/rfc7231#section-6.5.3"
    },
    {
      "title": "MDN: HTTP authentication",
      "url": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Guides/Authentication"
    },
    {
      "title": "Cloudflare: Verified bots",
      "url": "https://developers.cloudflare.com/bots/concepts/bot/verified-bots/"
    },
    {
      "title": "Cloudflare: Web Bot Auth",
      "url": "https://developers.cloudflare.com/bots/reference/bot-verification/web-bot-auth/"
    },
    {
      "title": "RFC 9421: HTTP Message Signatures",
      "url": "https://www.rfc-editor.org/rfc/rfc9421"
    },
    {
      "title": "Agent credentials on Rightshold",
      "url": "https://www.rightshold.com/agent-credentials"
    },
    {
      "title": "Agent gate on Rightshold",
      "url": "https://www.rightshold.com/agent-gate"
    }
  ],
  "related": [
    {
      "title": "How to require a verified agent credential before an API returns paid or sensitive data",
      "url": "https://www.rightshold.com/answers/how-to-require-a-verified-agent-credential-before-an-api-returns-paid-or-sensitive-data"
    },
    {
      "title": "How to set per-agent rate limits and quotas that do not throttle human browsers",
      "url": "https://www.rightshold.com/answers/how-to-set-per-agent-rate-limits-and-quotas-that-do-not-throttle-human-browsers"
    },
    {
      "title": "How to verify which AI agent is hitting your site with Web Bot Auth",
      "url": "https://www.rightshold.com/answers/how-to-verify-which-ai-agent-is-hitting-your-site-with-web-bot-auth"
    },
    {
      "title": "How to give an AI agent a scoped, expiring credential instead of a long-lived API key",
      "url": "https://www.rightshold.com/answers/how-to-give-an-ai-agent-a-scoped-expiring-credential-instead-of-a-long-lived-api-key"
    }
  ],
  "computed_from": "Curated guidance; license verdicts computed from Rightshold's license profiles at render time. Not legal advice.",
  "updated": "2026-11-07"
}