Google · Search engines · json
User-agent: Googlebot Disallow: /
| robots.txt token | Googlebot |
| User-agent contains | Googlebot |
| Operator | |
| Category | Search engines |
| robots.txt | obeys robots.txt (documented) |
| Verify by | published IP ranges |
| Published ranges | 169 IPv4 + 146 IPv6 · json · source |
The classic search crawler. It is also the crawler behind AI Overviews: Google does not run a separate bot for them, which is why the only AI opt-out is the Google-Extended token and not a Googlebot block.
Total. You leave Google Search. Never block this to avoid AI use; use Google-Extended instead.
Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)
User-agent: Googlebot Allow: /
Operator documentation: https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers
Machine copies: json ·
markdown
Policies that name this crawler:
allow-all · allow-ai-search-only · maximum-ai-visibility