8 crawler(s) · json
| Crawler | Token | Category | robots.txt |
|---|---|---|---|
| Google-CloudVertexBot | Google-CloudVertexBot | AI search crawlers | obeys robots.txt (documented) |
| Google-Extended | Google-Extended | AI training crawlers | control token only — no crawler |
| Google-InspectionTool | Google-InspectionTool | Tools and frameworks | obeys robots.txt (documented) |
| Googlebot | Googlebot | Search engines | obeys robots.txt (documented) |
| Googlebot-Image | Googlebot-Image | Search engines | obeys robots.txt (documented) |
| Googlebot-News | Googlebot-News | Search engines | obeys robots.txt (documented) |
| GoogleOther | GoogleOther | AI training crawlers | obeys robots.txt (documented) |
| Storebot-Google | Storebot-Google | Search engines | obeys robots.txt (documented) |
| Endpoint | IPv4 | IPv6 | Operator timestamp |
|---|---|---|---|
| Googlebot | 169 | 146 | 2026-08-31T14:46:46.000000 |
| Google special-purpose crawlers | 135 | 135 | 2026-08-31T14:46:39.000000 |
| Google user-triggered fetchers | 528 | 528 | 2026-08-31T14:46:36.000000 |
| Google user-triggered fetchers (Google-owned ranges) | 247 | 247 | 2026-08-31T14:46:39.000000 |
Documentation: https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers