curl -s https://www.pathwren.workers.dev/policy/index.json   # this page, as JSON

No key, no account, no handshake — every page here has a JSON twin one hop away. Machine doors: 6 keyless GET tools · documents.json · changes · llms.txt · openapi.json · agent card · mcp · a2a

Ready-made robots.txt

Each file names every relevant crawler explicitly, so a later change is a one-line diff and the intent stays readable. Pick the trade-off, curl the file.

curl -s https://www.pathwren.workers.dev/robots/block-ai-training.txt
PolicyCrawlers namedFile
Allow everything, explicitly150.txt
Block AI training, keep AI search27.txt
Block every AI crawler77.txt
Block corpus and dataset builders17.txt
Allow AI search and user fetches, block the rest65.txt
Block SEO and backlink crawlers17.txt
Block the crawlers with disputed robots compliance18.txt
Maximum AI visibility133.txt

These files get read by people, and it is measured. In the 24 hours to 2026-09-06T06:16:16+00:00, 169 readers this project never planted arrived from a link somebody else posted and took 124 requests across these policy pages and the robots.txt files themselves. What that day contained, in full, with the SQL behind every figure: 24 hours of crawler and agent traffic against this host.

Measured here, not asserted: 24 hours of AI-crawler traffic, as the actual table — what actually arrives at a small site that publishes these files: 1,267 of 3,374 client keys announced themselves as crawlers in one named window, and the biggest single fleet wore 352 addresses under one string. One of five documents about the same 24 hours — the other four are named at the foot of each one — every one of them also at .md and .json, with the figures and the SQL under the data index beside them.

This page as markdown: /c/mbin/policy/index.md — the same text, no markup to strip, no JavaScript, no key, CC0. Every page here has one: add .md to any address (also .mdx, <page>.html.md, <page>.html.mdx), or send Accept: text/markdown to this one. All of them in a single index: /c/mbin/sitemap.md.