Rightshold™ Sign up

Answers · AI crawlers

What is DuckAssistBot, is it training AI on my site, and if I block it in robots.txt will my pages disappear from DuckDuckGo search results?

Last updated

TL;DR: DuckAssistBot is DuckDuckGo's crawler for Search Assist, the short AI-generated answers that sit in DuckDuckGo results and link to one or two sources. DuckDuckGo says it fetches pages in real time for those answers and that the data isn't used in any way to train AI models. Blocking it is a separate switch from DuckDuckGo search: DuckDuckGo says opting out doesn't affect organic rankings or whether your site appears in its results, and that the change takes effect 72 hours after you disallow it in robots.txt. Its regular crawler is DuckDuckBot, and DuckDuckGo says its traditional links come largely from Bing, so neither DuckAssistBot nor DuckDuckBot alone decides whether you show up as a link. Before blocking, weigh what you lose: Search Assist always cites its sources, so a block removes a citation path rather than a training feed. To verify a request really is DuckAssistBot, match the source IP against duckduckgo.com/duckassistbot.json; the user-agent string can be typed by anyone.

What should I check, step by step?

1. Know which bot is which: DuckAssistBot fetches pages for Search Assist answers; DuckDuckBot is DuckDuckGo's general crawler. A rule for one doesn't apply to the other.

  1. To opt out of being a Search Assist source only, add "User-agent: DuckAssistBot" followed by "Disallow: /". DuckDuckGo says the change takes effect after 72 hours and doesn't affect organic rankings or inclusion in its results.
  2. Don't block it just to stop AI training. DuckDuckGo states DuckAssistBot data "is not used in any way to train AI models", so blocking it only removes you as a cited source in Search Assist.
  3. If a broad rule in your * group disallows unknown bots, add an explicit DuckAssistBot group with Allow: lines for the sections you want cited. A crawler follows only the most specific group that names it.
  4. If you want out of DuckDuckGo's regular links, DuckAssistBot isn't the lever. DuckDuckGo says it sources traditional links largely from Bing and also runs DuckDuckBot.
  5. Verify claimed DuckAssistBot traffic against duckduckgo.com/duckassistbot.json before treating it as DuckDuckGo. On November 11, 2026 the list held 486 IPv4 /32 prefixes, with a creation time of September 1, 2026.
  6. Expect 72 hours after a change before judging whether it worked. If DuckAssistBot still fetches disallowed pages after that, DuckDuckGo gives crawling@duckduckgo.com for opt-out problems; keep dated copies of your robots.txt and the log lines.

What is DuckAssistBot, and does DuckDuckGo use it to train AI?

DuckDuckGo's help page (checked November 11, 2026) describes DuckAssistBot as a web crawler for DuckDuckGo Search that crawls pages in real time for its AI-assisted answers, which prominently cite their sources. The same page says: "This data is not used in any way to train AI models." Its user agent appears as "DuckAssistBot/1.2; (+http://duckduckgo.com/duckassistbot.html)".

Does blocking DuckAssistBot remove your site from DuckDuckGo search results?

DuckDuckGo says publishers can opt out of being a potential source for AI-assisted answers by disallowing the DuckAssistBot user agent in robots.txt, that the change takes effect after 72 hours, and that opting out "does not impact organic search rankings and will not affect whether or not websites appear in our search results."

How do you verify a request really comes from DuckAssistBot?

DuckDuckGo publishes DuckAssistBot's addresses at duckduckgo.com/duckassistbot.json. Fetched on November 11, 2026, the file held 486 IPv4 prefixes, all single addresses (/32), with a creation time of September 1, 2026. DuckDuckGo documents no reverse DNS pattern and no request signature for it, so the IP list is the check.

How do I ask Rightshold about this (free, no key)?

Add the MCP server https://www.rightshold.com/mcp to Claude, Cursor, ChatGPT, Grok or VS Code, then ask: “Before we add a DuckAssistBot rule, what does our robots.txt tell the AI search and user-fetch crawlers today, and which of them fall through to our * group?”

One-click setup

{
  "tool": "read_ai_policy",
  "arguments": {
    "domain": "example.com"
  }
}

Rightshold reads a site's public crawl and AI-use signals. It isn't affiliated with DuckDuckGo, can't see what DuckDuckGo has crawled or cited and can't make any company honor a preference. Crawler descriptions, user-agent strings and the IP list are as DuckDuckGo published them, and the robots.txt files described were fetched, on November 11, 2026; they may change. Not legal advice.

What are the sources?

Related

Not legal advice. JSON · Markdown · All answers

Get a free API key in one call

No card and no email needed. 100 free credits; the license and policy checks stay free without a key.

curl -s -X POST https://www.rightshold.com/v1/accounts/signup -H "Content-Type: application/json" -d '{"accept_terms":true}'
export RIGHTSHOLD_API_KEY=rs_...   # the api_key from the reply (shown once)
curl -s https://www.rightshold.com/v1/licenses/check -H "Authorization: Bearer $RIGHTSHOLD_API_KEY" \
  -H "Content-Type: application/json" -d '{"license":"MIT","intended_use":{"commercial_use":true}}'