Claude-SearchBot

Anthropic · AI search crawlers · json

User-agent: Claude-SearchBot
Disallow: /


robots.txt token	`Claude-SearchBot`
User-agent contains	`Claude-SearchBot`
Operator	Anthropic
Category	AI search crawlers
robots.txt	obeys robots.txt (documented)
Verify by	no published verification method

What it is

Indexes pages so Claude's web search can find and cite them. Separate token from the training crawler, so search visibility and training consent are independent decisions.

What blocking it costs you

You stop appearing in Claude's search results and citations.

Full user-agent string

Mozilla/5.0 (compatible; Claude-SearchBot/1.0; +Claude-SearchBot@anthropic.com)

Allow it instead

User-agent: Claude-SearchBot
Allow: /

Operator documentation: https://support.anthropic.com/en/articles/8896518
Machine copies: json · markdown
Policies that name this crawler: allow-all · block-all-ai · allow-ai-search-only · maximum-ai-visibility