What this client asked www.pathwren.workers.dev for, when, and what it got.
Every number below is from this host's own server-side request log, 2026-08-31 20:58:34Z to 2026-09-01 03:00:09Z UTC.
Mozilla/5.0 (compatible; archive.org_bot +http://archive.org/details/archive.org_bot) Zeno/d3712ae warc/v0.8.99
327 request(s) from 2 distinct address(es), 270 distinct path(s), first seen 2026-08-31 20:58:34Z, last seen 2026-09-01 03:00:09Z UTC. 2 separate visit(s), counting a gap of more than 30 minutes as a new one, median 361.5 minutes between them.
It describes itself, inside its own user-agent, as archive.org_bot +http://archive.org/details/archive.org_bot
. That is the client's own words, quoted; this page makes no claim about what it is for.
The crawler catalogue on this site has a record for it: /crawler/archive-org-bot.html — what it is for, and what blocking it costs. This page is only what it did here.
| user-agent | requests | addresses | first seen | last seen |
|---|---|---|---|---|
Mozilla/5.0 (compatible; archive.org_bot +http://archive.org/details/archive.org_bot) Zeno/d3712ae warc/v0.8.99 | 327 | 2 | 2026-08-31 20:58:34Z | 2026-09-01 03:00:09Z |
First request to each path, oldest first, first 40 shown.
| # | path | at (UTC) | status |
|---|---|---|---|
| 1 | /category/dataset.html | 2026-08-31 20:58:34Z | not recorded |
| 2 | /category/archive.html | 2026-08-31 20:58:34Z | not recorded |
| 3 | /robots/block-disputed.txt | 2026-08-31 20:58:34Z | not recorded |
| 4 | /crawler/oai-searchbot.html | 2026-08-31 20:58:34Z | not recorded |
| 5 | /status.html | 2026-08-31 20:58:34Z | not recorded |
| 6 | /robots/allow-ai-search-only.txt | 2026-08-31 20:58:34Z | not recorded |
| 7 | /category/user-fetch.html | 2026-08-31 20:58:34Z | not recorded |
| 8 | /data/ | 2026-08-31 20:58:34Z | not recorded |
| 9 | /robots/maximum-ai-visibility.txt | 2026-08-31 20:58:34Z | not recorded |
| 10 | /crawler/ccbot.html | 2026-08-31 20:58:34Z | not recorded |
| 11 | /category/seo.html | 2026-08-31 20:58:34Z | not recorded |
| 12 | /policy/block-disputed.html | 2026-08-31 20:58:34Z | not recorded |
| 13 | /category/tool.html | 2026-08-31 20:58:34Z | not recorded |
| 14 | /crawler/gptbot.html | 2026-08-31 20:58:34Z | not recorded |
| 15 | /crawler/perplexitybot.html | 2026-08-31 20:58:34Z | not recorded |
| 16 | /snippet/ | 2026-08-31 20:58:34Z | not recorded |
| 17 | /ip-ranges/ | 2026-08-31 20:58:34Z | not recorded |
| 18 | /policy/allow-ai-search-only.html | 2026-08-31 20:58:34Z | not recorded |
| 19 | /crawler/claude-searchbot.html | 2026-08-31 20:58:34Z | not recorded |
| 20 | /policy/block-all-ai.html | 2026-08-31 20:58:34Z | not recorded |
| 21 | /ip-ranges/index.html | 2026-08-31 20:58:34Z | not recorded |
| 22 | /px.gif | 2026-08-31 20:58:35Z | not recorded |
| 23 | /data/agents.json | 2026-08-31 20:58:35Z | not recorded |
| 24 | /data/index.html | 2026-08-31 20:58:39Z | not recorded |
| 25 | /ip-ranges/openai-chatgpt-user.json | 2026-08-31 20:58:41Z | not recorded |
| 26 | /crawler/chatgpt-user.html | 2026-08-31 20:58:41Z | not recorded |
| 27 | /crawler/gptbot.json | 2026-08-31 20:58:41Z | not recorded |
| 28 | /crawler/oai-searchbot.json | 2026-08-31 20:58:41Z | not recorded |
| 29 | /ip-ranges/openai-gptbot.json | 2026-08-31 20:58:41Z | not recorded |
| 30 | /ip-ranges/openai-searchbot.json | 2026-08-31 20:58:41Z | not recorded |
| 31 | /crawler/chatgpt-user.json | 2026-08-31 20:58:41Z | not recorded |
| 32 | /crawler/claudebot.html | 2026-08-31 20:58:41Z | not recorded |
| 33 | /data/Mozilla/5.0%20(compatible;%20ClaudeBot/1.0;%20+claudebot@anthropic.com) | 2026-08-31 20:58:43Z | not recorded |
| 34 | /crawler/claudebot.json | 2026-08-31 20:58:43Z | not recorded |
| 35 | /crawler/claude-searchbot.json | 2026-08-31 20:58:43Z | not recorded |
| 36 | /data/Mozilla/5.0%20(compatible;%20Claude-SearchBot/1.0;%20+Claude-SearchBot@anthropic.com) | 2026-08-31 20:58:43Z | not recorded |
| 37 | /data/Mozilla/5.0%20(compatible;%20Claude-User/1.0;%20+Claude-User@anthropic.com) | 2026-08-31 20:58:43Z | not recorded |
| 38 | /crawler/claude-user.html | 2026-08-31 20:58:43Z | not recorded |
| 39 | /crawler/claude-user.json | 2026-08-31 20:58:43Z | not recorded |
| 40 | /crawler/anthropic-ai.html | 2026-08-31 20:58:43Z | not recorded |
| path | requests | status codes | first | last |
|---|---|---|---|---|
/px.gif | 51 | not recorded×45, 200×6 | 2026-08-31 20:58:35Z | 2026-09-01 03:00:09Z |
/crawler/bytespider.html | 2 | not recorded×1, 200×1 | 2026-08-31 20:59:01Z | 2026-09-01 03:00:04Z |
/crawler/ccbot.html | 2 | not recorded×2 | 2026-08-31 20:58:34Z | 2026-08-31 20:59:01Z |
/crawler/claude-searchbot.html | 2 | not recorded×2 | 2026-08-31 20:58:34Z | 2026-08-31 20:58:43Z |
/crawler/gptbot.html | 2 | not recorded×2 | 2026-08-31 20:58:34Z | 2026-08-31 20:58:41Z |
/crawler/oai-searchbot.html | 2 | not recorded×2 | 2026-08-31 20:58:34Z | 2026-08-31 20:58:41Z |
/crawler/perplexitybot.html | 2 | not recorded×2 | 2026-08-31 20:58:34Z | 2026-08-31 20:58:59Z |
/openapi.json | 2 | not recorded×1, 200×1 | 2026-08-31 20:59:36Z | 2026-09-01 03:00:04Z |
/about.html | 1 | not recorded×1 | 2026-08-31 20:59:44Z | 2026-08-31 20:59:44Z |
/apple-touch-icon.png | 1 | 200×1 | 2026-09-01 03:00:05Z | 2026-09-01 03:00:05Z |
/category/ai-search.html | 1 | 200×1 | 2026-09-01 03:00:04Z | 2026-09-01 03:00:04Z |
/category/ai-training.html | 1 | 200×1 | 2026-09-01 03:00:04Z | 2026-09-01 03:00:04Z |
/category/archive.html | 1 | not recorded×1 | 2026-08-31 20:58:34Z | 2026-08-31 20:58:34Z |
/category/dataset.html | 1 | not recorded×1 | 2026-08-31 20:58:34Z | 2026-08-31 20:58:34Z |
/category/seo.html | 1 | not recorded×1 | 2026-08-31 20:58:34Z | 2026-08-31 20:58:34Z |
/category/tool.html | 1 | not recorded×1 | 2026-08-31 20:58:34Z | 2026-08-31 20:58:34Z |
/category/user-fetch.html | 1 | not recorded×1 | 2026-08-31 20:58:34Z | 2026-08-31 20:58:34Z |
/crawler/DuckDuckBot/1.1%3B%20(%2Bhttp%3A/duckduckgo.com/duckduckbot.html) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/DuckDuckBot/1.1;%20(+http://duckduckgo.com/duckduckbot.html) | 1 | not recorded×1 | 2026-08-31 20:59:43Z | 2026-08-31 20:59:43Z |
/crawler/Googlebot-Image/1.0 | 1 | not recorded×1 | 2026-08-31 20:59:42Z | 2026-08-31 20:59:42Z |
/crawler/Mozilla/5.0%20(Linux%3B%20Android%205.0)%20AppleWebKit/537.36%20(KHTML%2C%20like%20Gecko)%20Mobile%20Safari/537.36%20(compatible%3B%20Bytespider%3B%20spider-feedback%40bytedance.com) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(Linux;%20Android%205.0)%20AppleWebKit/537.36%20(KHTML,%20like%20Gecko)%20Mobile%20Safari/537.36%20(compatible;%20Bytespider;%20spider-feedback@bytedance.com) | 1 | not recorded×1 | 2026-08-31 20:59:42Z | 2026-08-31 20:59:42Z |
/crawler/Mozilla/5.0%20(compatible%3B%20Baiduspider/2.0%3B%20%2Bhttp%3A/www.baidu.com/search/spider.html) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20Claude-SearchBot/1.0%3B%20%2BClaude-SearchBot%40anthropic.com) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20Claude-User/1.0%3B%20%2BClaude-User%40anthropic.com) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20Claude-Web/1.0) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20ClaudeBot/1.0%3B%20%2Bclaudebot%40anthropic.com) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20Diffbot/0.1%3B%20%2Bhttp%3A/www.diffbot.com) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20FirecrawlAgent/1.0%3B%20%2Bhttps%3A/firecrawl.dev) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20Google-InspectionTool/1.0%3B) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20GoogleOther) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20Googlebot/2.1%3B%20%2Bhttp%3A/www.google.com/bot.html) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20ImagesiftBot%3B%20%2Bimagesift.com) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20SemrushBot-OCOB/1.0%3B%20%2Bhttp%3A/www.semrush.com/bot.html) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20SemrushBot/7~bl%3B%20%2Bhttp%3A/www.semrush.com/bot.html) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20TikTokSpider%3B%20ttspider-feedback%40tiktok.com) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20Timpibot/0.1%3B%20%2Bhttps%3A/timpi.io) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20Webzio-Extended/1.0) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20YouBot%20(%2Bhttp%3A/www.you.com)) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20archive.org_bot%20%2Bhttp%3A/archive.org/details/archive.org_bot) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20bingbot/2.0%3B%20%2Bhttp%3A/www.bing.com/bingbot.htm) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20cohere-ai) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible%3B%20cohere-training-data-crawler) | 1 | not recorded×1 | 2026-08-31 20:59:48Z | 2026-08-31 20:59:48Z |
/crawler/Mozilla/5.0%20(compatible;%20Baiduspider/2.0;%20+http://www.baidu.com/search/spider.html) | 1 | not recorded×1 | 2026-08-31 20:59:43Z | 2026-08-31 20:59:43Z |
/crawler/Mozilla/5.0%20(compatible;%20Claude-SearchBot/1.0;%20+Claude-SearchBot@anthropic.com) | 1 | not recorded×1 | 2026-08-31 20:59:42Z | 2026-08-31 20:59:42Z |
/crawler/Mozilla/5.0%20(compatible;%20Claude-User/1.0;%20+Claude-User@anthropic.com) | 1 | not recorded×1 | 2026-08-31 20:59:42Z | 2026-08-31 20:59:42Z |
/crawler/Mozilla/5.0%20(compatible;%20Claude-Web/1.0) | 1 | not recorded×1 | 2026-08-31 20:59:42Z | 2026-08-31 20:59:42Z |
/crawler/Mozilla/5.0%20(compatible;%20ClaudeBot/1.0;%20+claudebot@anthropic.com) | 1 | not recorded×1 | 2026-08-31 20:59:42Z | 2026-08-31 20:59:42Z |
/crawler/Mozilla/5.0%20(compatible;%20Diffbot/0.1;%20+http://www.diffbot.com) | 1 | not recorded×1 | 2026-08-31 20:59:43Z | 2026-08-31 20:59:43Z |
/crawler/Mozilla/5.0%20(compatible;%20FirecrawlAgent/1.0;%20+https://firecrawl.dev) | 1 | not recorded×1 | 2026-08-31 20:59:44Z | 2026-08-31 20:59:44Z |
/crawler/Mozilla/5.0%20(compatible;%20Google-InspectionTool/1.0;) | 1 | not recorded×1 | 2026-08-31 20:59:42Z | 2026-08-31 20:59:42Z |
/crawler/Mozilla/5.0%20(compatible;%20GoogleOther) | 1 | not recorded×1 | 2026-08-31 20:59:42Z | 2026-08-31 20:59:42Z |
/crawler/Mozilla/5.0%20(compatible;%20Googlebot/2.1;%20+http://www.google.com/bot.html) | 1 | not recorded×1 | 2026-08-31 20:59:42Z | 2026-08-31 20:59:42Z |
/crawler/Mozilla/5.0%20(compatible;%20ImagesiftBot;%20+imagesift.com) | 1 | not recorded×1 | 2026-08-31 20:59:43Z | 2026-08-31 20:59:43Z |
/crawler/Mozilla/5.0%20(compatible;%20SemrushBot-OCOB/1.0;%20+http://www.semrush.com/bot.html) | 1 | not recorded×1 | 2026-08-31 20:59:43Z | 2026-08-31 20:59:43Z |
/crawler/Mozilla/5.0%20(compatible;%20SemrushBot/7~bl;%20+http://www.semrush.com/bot.html) | 1 | not recorded×1 | 2026-08-31 20:59:43Z | 2026-08-31 20:59:43Z |
/crawler/Mozilla/5.0%20(compatible;%20TikTokSpider;%20ttspider-feedback@tiktok.com) | 1 | not recorded×1 | 2026-08-31 20:59:42Z | 2026-08-31 20:59:42Z |
/crawler/Mozilla/5.0%20(compatible;%20Timpibot/0.1;%20+https://timpi.io) | 1 | not recorded×1 | 2026-08-31 20:59:43Z | 2026-08-31 20:59:43Z |
/crawler/Mozilla/5.0%20(compatible;%20Webzio-Extended/1.0) | 1 | not recorded×1 | 2026-08-31 20:59:43Z | 2026-08-31 20:59:43Z |
/crawler/Mozilla/5.0%20(compatible;%20YouBot%20(+http://www.you.com)) | 1 | not recorded×1 | 2026-08-31 20:59:43Z | 2026-08-31 20:59:43Z |
210 further paths are in the JSON.
Nothing. Every path it asked for existed at the time it asked.
no published verification method
From the crawler record at /crawler/archive-org-bot.html, which cites https://archive.org/details/archive.org_bot — one source for both pages.
Not observed. We have not seen this client publish a grade, a listing or a record about this host anywhere, and we make no claim that it does or does not.
It names http://archive.org/details/archive.org_bot in its own user-agent. That is the operator's own claim about itself; we neither endorse nor verify what is on it.
curl -s https://www.pathwren.workers.dev/bot/archive-org-bot.json curl -s https://www.pathwren.workers.dev/data/observed-clients.json # every client, one request
JSON · markdown · all clients seen here · index as JSON
Rows come from a server-side log written before anything is served, so clients that
run no JavaScript are counted exactly like browsers. The window is the whole life of the log,
2026-08-31 20:58:34Z to 2026-09-01 03:00:09Z UTC, and it is stated on every number because a count without a window is
not a fact. Addresses are stored as salted hashes and counted, never printed. Our own checks
send X-Self: 1 and are excluded, together with every request the analyst flagged
as one of our own agents reaching for a URL through a model provider's fetch tool
(the numbers are on the index). Nothing here is a claim about
intent: where this page does not know something it says not observed
.