curl -s https://www.pathwren.workers.dev/data/index.json   # this page, as JSON

No key, no account, no handshake — every page here has a JSON twin one hop away. Machine doors: 6 keyless GET tools · documents.json · changes · llms.txt · openapi.json · agent card · mcp · a2a

Bulk data

Static files. No key, no signup, no rate limit, CORS open, and every one of them is the same file the HTML pages are generated from.

curl -s https://www.pathwren.workers.dev/data/agents.json | jq '.crawlers[] | select(.category=="ai-search") | .robots_token'
curl -s https://www.pathwren.workers.dev/ip-ranges/all.txt | wc -l
curl -s https://www.pathwren.workers.dev/robots/block-ai-training.txt >> robots.txt

Endpoint	What it is	Size
/data/agents.json	Every record, with the endpoint map and category definitions.	150 records
/data/agents.csv	The same table, flat, for a spreadsheet or a COPY INTO.	150 rows
/data/user-agents.txt	User-agent substrings, one per line. Feed to grep.	148 lines
/data/robots-tokens.txt	robots.txt tokens, one per line.	149 lines
/data/ua-regex.json	Pre-escaped alternations, whole-list and per category.	—
/data/ua-regex.txt	The single all-crawlers regex, nothing else.	1 line
/data/ip-sources.json	Which operators publish IP ranges, and where.	15 sources
/ip-ranges/all.json	Union of every published prefix, grouped by source.	1987+1062 prefixes
/ip-ranges/all.txt	The same prefixes, one CIDR per line, for a WAF.	3049 lines
/status.json	Freshness and health of every upstream endpoint.	—
/openapi.json	OpenAPI 3.1 description of all of the above.	—
/llms-full.txt	The whole index as one markdown file.	—
/feed.json	JSON Feed 1.1: what changed.	—
/feed.xml	RSS 2.0, same content.	—

This page as markdown: /data/index.md — the same text, no markup to strip, no JavaScript, no key, CC0. Every page here has one: add .md to any address (also .mdx, <page>.html.md, <page>.html.mdx), or send Accept: text/markdown to this one. All of them in a single index: /sitemap.md.