| Ahrefs | 1 | SEO and backlink crawlers |
| Allen Institute for AI | 2 | Corpus and dataset builders |
| Amazon | 1 | AI search crawlers |
| Anthropic | 5 | AI search crawlers, AI training crawlers, User-triggered fetchers |
| Apple | 2 | AI training crawlers, Search engines |
| Baidu | 1 | Search engines |
| ByteDance | 2 | AI training crawlers |
| Cohere | 2 | AI training crawlers, User-triggered fetchers |
| Common Crawl | 1 | Corpus and dataset builders |
| Diffbot | 1 | Corpus and dataset builders |
| DuckDuckGo | 2 | AI search crawlers, Search engines |
| Firecrawl | 1 | Tools and frameworks |
| Google | 8 | AI search crawlers, AI training crawlers, Search engines, Tools and frameworks |
| Hive AI | 1 | Corpus and dataset builders |
| Huawei | 1 | Search engines |
| Internet Archive | 2 | Archivers |
| LAION / img2dataset | 1 | Corpus and dataset builders |
| Meta | 4 | AI training crawlers, Link preview fetchers, User-triggered fetchers |
| Microsoft | 1 | Search engines |
| Mistral AI | 1 | User-triggered fetchers |
| Naver | 1 | Search engines |
| OpenAI | 3 | AI search crawlers, AI training crawlers, User-triggered fetchers |
| Perplexity | 2 | AI search crawlers, User-triggered fetchers |
| Scrapy project | 1 | Tools and frameworks |
| Semrush | 2 | AI training crawlers, SEO and backlink crawlers |
| Seznam | 1 | Search engines |
| Timpi | 1 | Search engines |
| Webz.io | 3 | AI training crawlers, Corpus and dataset builders |
| Yandex | 1 | Search engines |
| You.com | 1 | AI search crawlers |