robots.txt policy: Allow AI search and user fetches, block the rest

Be findable and citable in assistants without contributing to training corpora. The inverse framing of block-ai-training, written as an allowlist so the default for anything new is deny. Fetches a user explicitly asked for stay allowed, because refusing those produces a visible error for a real person who wanted your page. curl -s https://www.pathwren.workers.dev/robots/allow-ai-search-only.txt