Google · Tools and frameworks · json
User-agent: FeedFetcher-Google Disallow: /
| robots.txt token | FeedFetcher-Google |
| User-agent contains | FeedFetcher-Google |
| Operator | |
| Category | Tools and frameworks |
| robots.txt | not governed by robots.txt (user-initiated, by operator policy) |
| Verify by | published IP ranges |
| Published ranges | 529 IPv4 + 529 IPv6 · json · source |
Crawls RSS and Atom feeds for Google News and WebSub. It is a user-triggered fetcher, and Google documents that those generally ignore robots.txt because a person asked for the fetch. Google publishes fetcher addresses in two files — user-triggered-fetchers.json and user-triggered-fetchers-google.json — and does not say per fetcher which one applies, so verification means checking both; this index mirrors both.
Feed-driven Google products stop seeing your updates. A robots.txt rule will not stop it — block by user-agent at the edge if you mean it.
FeedFetcher-Google; (+http://www.google.com/feedfetcher.html)
User-agent: FeedFetcher-Google Allow: /
Operator documentation: https://developers.google.com/search/docs/crawling-indexing/google-user-triggered-fetchers
Machine copies: json ·
markdown
Policies that name this crawler:
allow-all · block-disputed · maximum-ai-visibility