curl -s https://www.pathwren.workers.dev/operator/google.json # this page, as JSON
No key, no account, no handshake — every page here has a JSON twin one hop away. Machine doors: 6 keyless GET tools · documents.json · changes · llms.txt · openapi.json · agent card · mcp · a2a
26 crawler(s) · json
| Crawler | Token | Category | robots.txt |
|---|---|---|---|
| AdsBot-Google | AdsBot-Google | Tools and frameworks | ignores the * group; obeys rules named for its own token |
| AdsBot-Google-Mobile | AdsBot-Google-Mobile | Tools and frameworks | ignores the * group; obeys rules named for its own token |
| AdsBot-Google-Mobile-Apps | AdsBot-Google-Mobile-Apps | Tools and frameworks | ignores the * group; obeys rules named for its own token |
| APIs-Google | APIs-Google | Tools and frameworks | ignores the * group; obeys rules named for its own token |
| FeedFetcher-Google | FeedFetcher-Google | Tools and frameworks | not governed by robots.txt (user-initiated, by operator policy) |
| Google-Agent | Google-Agent | User-triggered fetchers | not governed by robots.txt (user-initiated, by operator policy) |
| Google-CloudVertexBot | Google-CloudVertexBot | AI search crawlers | obeys robots.txt (documented) |
| Google-CWS | Google-CWS | Tools and frameworks | not governed by robots.txt (user-initiated, by operator policy) |
| Google-Extended | Google-Extended | AI training crawlers | control token only — no crawler |
| Google-GeminiNotebook | Google-GeminiNotebook | User-triggered fetchers | not governed by robots.txt (user-initiated, by operator policy) |
| Google-InspectionTool | Google-InspectionTool | Tools and frameworks | obeys robots.txt (documented) |
| Google-Pinpoint | Google-Pinpoint | User-triggered fetchers | not governed by robots.txt (user-initiated, by operator policy) |
| Google-Read-Aloud | Google-Read-Aloud | User-triggered fetchers | not governed by robots.txt (user-initiated, by operator policy) |
| Google-Safety | Google-Safety | Tools and frameworks | not governed by robots.txt (user-initiated, by operator policy) |
| Google-Site-Verification | Google-Site-Verification | Tools and frameworks | not governed by robots.txt (user-initiated, by operator policy) |
| Googlebot | Googlebot | Search engines | obeys robots.txt (documented) |
| Googlebot-Image | Googlebot-Image | Search engines | obeys robots.txt (documented) |
| Googlebot-News | Googlebot-News | Search engines | obeys robots.txt (documented) |
| Googlebot-Video | Googlebot-Video | Search engines | obeys robots.txt (documented) |
| GoogleMessages | GoogleMessages | Link preview fetchers | not governed by robots.txt (user-initiated, by operator policy) |
| GoogleOther | GoogleOther | AI training crawlers | obeys robots.txt (documented) |
| GoogleOther-Image | GoogleOther-Image | AI training crawlers | obeys robots.txt (documented) |
| GoogleOther-Video | GoogleOther-Video | AI training crawlers | obeys robots.txt (documented) |
| GoogleProducer | GoogleProducer | Tools and frameworks | not governed by robots.txt (user-initiated, by operator policy) |
| Mediapartners-Google | Mediapartners-Google | Tools and frameworks | ignores the * group; obeys rules named for its own token |
| Storebot-Google | Storebot-Google | Search engines | obeys robots.txt (documented) |
| Endpoint | IPv4 | IPv6 | Operator timestamp |
|---|---|---|---|
| Googlebot | 170 | 147 | 2026-09-11T14:46:08.000000 |
| Google special-purpose crawlers | 136 | 136 | 2026-09-11T14:46:08.000000 |
| Google user-triggered fetchers | 529 | 529 | 2026-09-11T14:46:07.000000 |
| Google user-triggered fetchers (Google-owned ranges) | 248 | 248 | 2026-09-11T14:46:08.000000 |
| Google-Agent (agents on Google infrastructure) | 3 | 1 | 2026-03-03T10:00:00.000000 |
A question about this page, answered by a call you can make right now — no key, no account, no JSON-RPC, nothing to assemble. The query string is already filled in with this page's own subject.
curl -s 'https://www.pathwren.workers.dev/tools/verification-methods?crawler=google&s=client-dossiers'
/tools/verification-methods?crawler=google/tools/verify-crawler?ip=192.178.4.0&ua=GooglebotEvery parameter above is a value the endpoint reads, not an address it visits: nothing is fetched on your behalf. Called without any parameter, each endpoint returns its own specification instead of an error. All of them.
Documentation: https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers
This page as markdown: /operator/google.md — the same text, no markup to strip, no JavaScript, no key, CC0. Every page here has one: add .md to any address (also .mdx, <page>.html.md, <page>.html.mdx), or send Accept: text/markdown to this one. All of them in a single index: /sitemap.md.