curl -s https://www.pathwren.workers.dev/mcp.json   # this page, as JSON

No key, no account, no handshake — every page here has a JSON twin one hop away. Machine doors: 6 keyless GET tools · documents.json · changes · llms.txt · openapi.json · agent card · mcp · a2a

MCP server

Connect it: one paste, one call

https://www.pathwren.workers.dev/mcp
no auth, read-only, public

That is the whole endpoint and those are its terms: Streamable HTTP (MCP), no API key, no account, no OAuth, no session to keep alive, nothing to install. Every tool is read-only, and none of them will fetch a URL on your behalf.

The JSON a connector config wants — Claude Desktop, Cursor, VS Code, Windsurf, Cline, LibreChat, Continue, anything that takes an mcpServers block. Complete as it stands; there is no field to fill in:

{
  "mcpServers": {
    "ai-crawler-index": {
      "type": "streamable-http",
      "url": "https://www.pathwren.workers.dev/mcp"
    }
  }
}

Claude Code takes one line instead:

claude mcp add --transport http ai-crawler-index https://www.pathwren.workers.dev/mcp

Then call whoami first. It takes no arguments at all, so there is nothing to invent and nothing to look up before you can see this server work — the subject of the answer is the request you just sent:

curl -s https://www.pathwren.workers.dev/mcp \
  -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whoami","arguments":{}}}' \
  | jq -r '.result.content[0].text' | head -3

You are calling as: curl/8.7.1
From: 203.0.113.7 (CF-Connecting-IP)
Our own instrument books that user-agent as: agent.

Those are the answer's own first three lines. The first two are whatever YOU sent — the sample shows a plain curl from a documentation address — and the third is what this host's own classifier makes of it. The rest of the answer adds the index's own record of the user-agent you sent, whether we have seen you here before, and what our robots policy says to your user-agent. Nothing is fetched to answer it, no argument exists, and the same answer is a plain GET at /tools/whoami. example is the other zero-argument tool on this server: it runs this server's own worked example for real and hands back exactly what a real call returns. Either one is a safe first call.

The whole index, spoken as Model Context Protocol instead of as files. Streamable HTTP, stateless, no key, no signup, no session to keep alive. Every tool is read-only.

# it answers right now — no account, no header ceremony
curl -s https://www.pathwren.workers.dev/mcp \
  -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' \
  -d '{"jsonrpc":"2.0","id":1,"method":"tools/list"}' | jq -r '.result.tools[].name'

Tools

Tool	What it answers
`classify_user_agent`	Given a raw User-Agent header: which crawler is this, who runs it, what is it for, and what does blocking it cost me?
`lookup_crawler`	The full record for one crawler by slug, name or robots.txt token.
`list_crawlers`	The 150 crawlers, filtered by category, operator, robots.txt stance or verification method. Returns the valid filter values too, so one call teaches the vocabulary.
`generate_robots_txt`	A ready-to-paste robots.txt for one of 8 stances, with the reasoning and the exact tokens it implies.
`is_verified_crawler_ip`	Is this IP inside a range the operator itself publishes? 1987 IPv4 and 1062 IPv6 prefixes, mirrored from 15 endpoints, refreshed every six hours.
`whats_changed`	Build time, per-source freshness, which sources changed or are failing, and the dated changelog. The cron check for anything that regenerates a config from this data.

Resources

Four, each pointing at the file the tools are computed from: agents.json, ua-regex.json, ip-ranges/all.json, status.json. Read them directly if you would rather do your own matching.

What it will not do

It has no side effects and no write path: it cannot edit your robots.txt, block anything, or fetch a URL for you. A user-agent match is a claim, not a proof — classify_user_agent says so on every answer, and is_verified_crawler_ip is the tool that turns a claim into evidence. Where an operator publishes no ranges, no tool here can verify it, and each one says which case you are in.

Five more servers on this host

Different jobs, no shared tool names — run all six or just the one you need. crawler-log-triage at /mcp/triage takes a whole access log and returns per-line verdicts, the impersonators, and a robots.txt or WAF ruleset for exactly what was in it. agent-discovery-doctor at /mcp/doctor turns the question around: which of the 22 discovery documents agents ask for — llms.txt, A2A agent card, owners.json, mcp.json — does a host actually serve, and who asks for each missing one. robots-policy-lint at /mcp/robots reads a robots.txt you paste and reports what it really does, by RFC 9309 rules. crawler-ip-verifier at /mcp/netcheck is the batch and prefix form of is_verified_crawler_ip: 500 addresses at a time, CIDR overlap arithmetic, and a firewall allowlist you can paste. mcp-endpoint-lint at /mcp/lint is the one aimed at MCP itself: give it another server's endpoint and it reports what the handshake negotiated, whether every advertised tool schema compiles, whether the error codes are the ones JSON-RPC requires, and what a client finds before it dials — scored out of 100.

Protocol versions 2025-06-18, negotiated per call. server/discover answers for clients on 2026-07-28, initialize for everyone else. Same data as /data/agents.json and /openapi.json — pick whichever your client speaks. Listed in the official MCP Registry as dev.workers.pathwren.www/ai-crawler-index.