ShapBot

What this client asked www.pathwren.workers.dev for, when, and what it got. Every number below is from this host's own server-side request log, 2026-09-02 02:31:06Z to 2026-09-02 02:34:43Z UTC.

Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.0

346 request(s) from 7 distinct address(es), 314 distinct path(s), first seen 2026-09-02 02:31:06Z, last seen 2026-09-02 02:34:43Z UTC. 1 separate visit(s), counting a gap of more than 30 minutes as a new one.

The crawler catalogue on this site has a record for it: /crawler/shapbot.html — what it is for, and what blocking it costs. This page is only what it did here.

The user-agent strings, exactly as they arrived

user-agentrequestsaddressesfirst seenlast seen
Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; ShapBot/0.1.034672026-09-02 02:31:06Z2026-09-02 02:34:43Z

What it asked for, in the order it asked

First request to each path, oldest first, first 40 shown.

#pathat (UTC)status
1/mcp-doctor.html2026-09-02 02:31:06Z200
2/mcp-triage.html2026-09-02 02:31:07Z200
3/2026-09-02 02:31:07Z200
4/data2026-09-02 02:31:07Z404
5/mcp-robots.html2026-09-02 02:31:07Z200
6/bot2026-09-02 02:31:07Z404
7/mcp-netcheck.html2026-09-02 02:31:10Z200
8/operator2026-09-02 02:31:12Z404
9/crawler2026-09-02 02:31:12Z404
10/px.gif2026-09-02 02:31:12Z200
11/favicon.ico2026-09-02 02:31:14Z200
12/policy2026-09-02 02:31:46Z404
13/status.html2026-09-02 02:32:16Z200
14/api.html2026-09-02 02:32:18Z200
15/mcp.html2026-09-02 02:32:19Z200
16/about.html2026-09-02 02:32:19Z200
17/register2026-09-02 02:32:19Z200
18/reference2026-09-02 02:32:20Z200
19/security.html2026-09-02 02:32:20Z200
20/privacy.html2026-09-02 02:32:20Z200
21/ip-ranges2026-09-02 02:32:21Z404
22/terms.html2026-09-02 02:32:21Z200
23/crawler/gptbot.html2026-09-02 02:32:21Z200
24/crawler/oai-searchbot.html2026-09-02 02:32:21Z200
25/crawler/claudebot.html2026-09-02 02:32:22Z200
26/crawler/claude-searchbot.html2026-09-02 02:32:22Z200
27/crawler/google-extended.html2026-09-02 02:32:22Z200
28/crawler/perplexitybot.html2026-09-02 02:32:23Z200
29/crawler/ccbot.html2026-09-02 02:32:23Z200
30/crawler/bytespider.html2026-09-02 02:32:23Z200
31/category/search.html2026-09-02 02:32:24Z200
32/category/ai-training.html2026-09-02 02:32:24Z200
33/crawler/index.html2026-09-02 02:32:24Z200
34/category/tool.html2026-09-02 02:32:25Z200
35/category/ai-search.html2026-09-02 02:32:25Z200
36/category/dataset.html2026-09-02 02:32:27Z200
37/category/seo.html2026-09-02 02:32:27Z200
38/category/user-fetch.html2026-09-02 02:32:27Z200
39/category/preview.html2026-09-02 02:32:28Z200
40/category/archive.html2026-09-02 02:32:28Z200

Everything it asked for

pathrequestsstatus codesfirstlast
/favicon.ico7200×72026-09-02 02:31:14Z2026-09-02 02:34:43Z
/px.gif6200×62026-09-02 02:31:12Z2026-09-02 02:33:19Z
/.well-known/oauth-authorization-server2404×22026-09-02 02:33:30Z2026-09-02 02:34:35Z
/bot2404×22026-09-02 02:31:07Z2026-09-02 02:32:19Z
/crawler2404×22026-09-02 02:31:12Z2026-09-02 02:31:30Z
/crawler/anomura.html2200×22026-09-02 02:33:13Z2026-09-02 02:33:14Z
/crawler/ccbot.md2200×22026-09-02 02:33:32Z2026-09-02 02:33:33Z
/crawler/googleother-image.html2200×22026-09-02 02:32:50Z2026-09-02 02:32:51Z
/crawler/isscyberriskcrawler.html2200×22026-09-02 02:33:11Z2026-09-02 02:33:12Z
/crawler/quillbot.html2200×22026-09-02 02:33:12Z2026-09-02 02:33:13Z
/crawler/serpstatbot.html2200×22026-09-02 02:33:05Z2026-09-02 02:33:06Z
/crawler/slackbot-linkexpanding.html2200×22026-09-02 02:33:07Z2026-09-02 02:33:08Z
/data2404×22026-09-02 02:31:07Z2026-09-02 02:31:11Z
/ip-ranges2404×22026-09-02 02:32:21Z2026-09-02 02:33:17Z
/ip-ranges/all.txt2200×22026-09-02 02:33:31Z2026-09-02 02:33:32Z
/ip-ranges/bing-bingbot.txt2200×22026-09-02 02:33:32Z2026-09-02 02:33:33Z
/operator2404×22026-09-02 02:31:12Z2026-09-02 02:32:01Z
/operator/anthropic.html2200×22026-09-02 02:33:17Z2026-09-02 02:33:18Z
/operator/ceramic.html2200×22026-09-02 02:33:21Z2026-09-02 02:33:22Z
/operator/commoncrawl.html2200×22026-09-02 02:33:20Z2026-09-02 02:33:21Z
/operator/internetarchive.html2200×22026-09-02 02:33:23Z2026-09-02 02:33:24Z
/operator/webz.html2200×22026-09-02 02:33:28Z2026-09-02 02:33:29Z
/policy2404×22026-09-02 02:31:46Z2026-09-02 02:33:11Z
/1200×12026-09-02 02:31:07Z2026-09-02 02:31:07Z
/.well-known/api-catalog1200×12026-09-02 02:32:35Z2026-09-02 02:32:35Z
/.well-known/api-onboarding1200×12026-09-02 02:32:35Z2026-09-02 02:32:35Z
/.well-known/security.txt1200×12026-09-02 02:33:30Z2026-09-02 02:33:30Z
/a2a.html1200×12026-09-02 02:33:31Z2026-09-02 02:33:31Z
/about.html1200×12026-09-02 02:32:19Z2026-09-02 02:32:19Z
/api.html1200×12026-09-02 02:32:18Z2026-09-02 02:32:18Z
/bot/apievangelist.html1200×12026-09-02 02:33:16Z2026-09-02 02:33:16Z
/bot/archive-org-bot.html1200×12026-09-02 02:32:29Z2026-09-02 02:32:29Z
/bot/claudebot.html1200×12026-09-02 02:32:29Z2026-09-02 02:32:29Z
/bot/claudebot.md1200×12026-09-02 02:33:33Z2026-09-02 02:33:33Z
/bot/gptbot.html1200×12026-09-02 02:32:29Z2026-09-02 02:32:29Z
/bot/node.html1200×12026-09-02 02:32:30Z2026-09-02 02:32:30Z
/bot/sentineloracle.html1200×12026-09-02 02:32:30Z2026-09-02 02:32:30Z
/bot/yandexbot.html1200×12026-09-02 02:32:30Z2026-09-02 02:32:30Z
/bot/yandexbot.md1200×12026-09-02 02:33:33Z2026-09-02 02:33:33Z
/category/ai-search.html1200×12026-09-02 02:32:25Z2026-09-02 02:32:25Z
/category/ai-training.html1200×12026-09-02 02:32:24Z2026-09-02 02:32:24Z
/category/archive.html1200×12026-09-02 02:32:28Z2026-09-02 02:32:28Z
/category/dataset.html1200×12026-09-02 02:32:27Z2026-09-02 02:32:27Z
/category/preview.html1200×12026-09-02 02:32:28Z2026-09-02 02:32:28Z
/category/search.html1200×12026-09-02 02:32:24Z2026-09-02 02:32:24Z
/category/seo.html1200×12026-09-02 02:32:27Z2026-09-02 02:32:27Z
/category/tool.html1200×12026-09-02 02:32:25Z2026-09-02 02:32:25Z
/category/user-fetch.html1200×12026-09-02 02:32:27Z2026-09-02 02:32:27Z
/crawler/adsbot-google-mobile-apps.html1200×12026-09-02 02:32:52Z2026-09-02 02:32:52Z
/crawler/adsbot-google-mobile.html1200×12026-09-02 02:32:52Z2026-09-02 02:32:52Z
/crawler/adsbot-google.html1200×12026-09-02 02:32:51Z2026-09-02 02:32:51Z
/crawler/ahrefsbot.html1200×12026-09-02 02:32:47Z2026-09-02 02:32:47Z
/crawler/ahrefssiteaudit.html1200×12026-09-02 02:33:04Z2026-09-02 02:33:04Z
/crawler/ai2bot-dolma.html1200×12026-09-02 02:33:31Z2026-09-02 02:33:31Z
/crawler/ai2bot.html1200×12026-09-02 02:32:43Z2026-09-02 02:32:43Z
/crawler/aihitbot.html1200×12026-09-02 02:33:14Z2026-09-02 02:33:14Z
/crawler/aiwebindex.html1200×12026-09-02 02:33:13Z2026-09-02 02:33:13Z
/crawler/amazonbot.html1200×12026-09-02 02:32:42Z2026-09-02 02:32:42Z
/crawler/andibot.html1200×12026-09-02 02:33:13Z2026-09-02 02:33:13Z
/crawler/anthropic-ai.html1200×12026-09-02 02:32:36Z2026-09-02 02:32:36Z

254 further paths are in the JSON.

What it asked for that did not exist

path it asked forwhat it gotfirst askedsince then
/data404×22026-09-02 02:31:07Zanswers 200 since 2026-09-02 03:05:05Z
/bot404×22026-09-02 02:31:07Zanswers 200 since 2026-09-02 03:05:05Z
/operator404×22026-09-02 02:31:12Zanswers 200 since 2026-09-02 03:05:05Z
/crawler404×22026-09-02 02:31:12Zanswers 200 since 2026-09-02 03:05:06Z
/policy404×22026-09-02 02:31:46Zanswers 200 since 2026-09-02 03:05:55Z
/ip-ranges404×22026-09-02 02:32:21Zanswers 200 since 2026-09-02 03:05:06Z
/.well-known/oauth-authorization-server404×22026-09-02 02:33:30Zstill absent — on purpose: No

What it got, and what it sent

Status codes
200×332, 404×14
Accept headers
text/markdown, text/html;q=0.9, */*;q=0.8, image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8, text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
Bytes served
242952
Attributed to a channel
none — it arrived at a plain path
Our instrument classed it
crawler×346 — that is our classifier's label from the user-agent, not the operator's, and it has been wrong before.

How to verify it is really them

no published verification method

From the crawler record at /crawler/shapbot.html, which cites https://docs.parallel.ai/features/crawler — one source for both pages.

What it publishes about you afterwards

Not observed. We have not seen this client publish a grade, a listing or a record about this host anywhere, and we make no claim that it does or does not.

Its own documentation

Not observed. It carries no URL and no contact address in its user-agent, so there is nowhere documented to ask what it is.

This page as data

curl -s https://www.pathwren.workers.dev/bot/shapbot.json
curl -s https://www.pathwren.workers.dev/data/observed-clients.json   # every client, one request

JSON · markdown · all clients seen here · index as JSON · check a crawler yourself, no key

Method, and what this page will not say

Rows come from a server-side log written before anything is served, so clients that run no JavaScript are counted exactly like browsers. The window is the whole life of the log, 2026-09-02 02:31:06Z to 2026-09-02 02:34:43Z UTC, and it is stated on every number because a count without a window is not a fact. Addresses are stored as salted hashes and counted, never printed. Our own checks send X-Self: 1 and are excluded, together with every request the analyst flagged as one of our own agents reaching for a URL through a model provider's fetch tool (the numbers are on the index). Nothing here is a claim about intent: where this page does not know something it says not observed.