{"name":"crawler-ip-verifier","title":"Crawler IP Verifier — MCP server","version":"1.0.0","transport":"streamable-http","transport_docs":"https://www.pathwren.workers.dev/mcp-transport.html","endpoint":"https://www.pathwren.workers.dev/mcp/netcheck","protocol_versions":["2026-07-28","2025-11-25","2025-06-18","2025-03-26","2024-11-05"],"stateless":true,"auth":"none — public, read-only, no key, no rate limit","what_it_is":"Address and prefix work against the operator-published IP ranges this host mirrors: batch verification of up to 500 addresses, containment and overlap arithmetic on a CIDR, paste-ready firewall allowlists in eight formats, per-crawler verification methods, and the mirror's own freshness.","different_from":"https://www.pathwren.workers.dev/mcp answers one address with a boolean. This server is the batch, the prefix arithmetic and the artifact: different input shapes, different outputs, and it is the one to call from a script. https://www.pathwren.workers.dev/mcp/triage does the same identification starting from log LINES rather than addresses.","try_it":"curl -s https://www.pathwren.workers.dev/mcp/netcheck -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"tools/call\",\"params\":{\"name\":\"verify_batch\",\"arguments\":{\"addresses\":[\"20.171.206.1\",\"203.0.113.9\"]}}}'","data_behind_it":"https://www.pathwren.workers.dev/ip-ranges/all.json","reverse_dns":"documented per crawler, never performed — this server makes no outbound request","call_this_first":{"tool":"whoami","takes_arguments":false,"invitation":"Takes no arguments. Safe to call. Deterministic. Touches no third party.","curl":"curl -s https://www.pathwren.workers.dev/mcp/netcheck -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' -d '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"tools/call\",\"params\":{\"name\":\"whoami\",\"arguments\":{}}}'","also":"example — the same, running this server's own worked example end to end.","every_other_tool":"Every other tool here also accepts {}: an empty call returns that tool's documented worked example, labelled as one, instead of an error."},"no_fetch_tool":"No tool here fetches a URL. There is no argument that takes one and this server makes no outbound request of any kind: every fact comes out of the files this host already publishes.","tools":[{"name":"verify_batch","title":"Verify many addresses at once","description":"Check up to 500 addresses against every mirrored operator prefix list in one call. Returns, per address, the matching prefix, the source that published it and when it was mirrored, plus a rollup by operator. Accepts {ip, claim} pairs to test a claimed identity against the ranges, and says plainly that a miss is not evidence of a fake. Example: addresses=[{ip:'66.249.66.1', claim:'Googlebot'}, {ip:'203.0.113.9', claim:'GPTBot'}] — the first matches a published Google prefix, the second matches nothing. Also callable without MCP, same implementation: GET https://www.pathwren.workers.dev/tools/verify-crawler?ip=66.249.66.1&ua=Googlebot&s=client-dossiers"},{"name":"lookup_prefix","title":"Prefix arithmetic against published ranges","description":"Give a CIDR and get every published operator prefix that contains it, is contained by it, or partially overlaps it — the check for 'does my network collide with a crawler range' and for auditing an allowlist somebody handed you. Give an operator or source slug instead and get everything that source publishes. Example: cidr='66.249.66.0/24'."},{"name":"export_ip_acl","title":"Paste-ready allowlist or denylist","description":"Turn a set of operators into a config you can paste: nginx geo, nginx allow/deny, Apache, HAProxy, a Cloudflare firewall expression, an ipset script, a Caddy matcher, or a plain CIDR list. Every export carries a provenance header naming each source URL and the mirror time, and reports the rule cost. Example: operators='all', format='cidr-list', action='allow'."},{"name":"verification_methods","title":"Can this crawler be verified at all, and how?","description":"Per crawler: whether the operator publishes IP ranges, documents reverse DNS, or publishes no verification method at all — with the exact source URL or the exact dig commands, and how fresh this mirror is. Call it before reading anything into an address that matched nothing. Reverse DNS is never run here; the command is handed back instead. Example: crawler='claudebot'. Also callable without MCP, same implementation: GET https://www.pathwren.workers.dev/tools/verification-methods?crawler=claudebot&s=client-dossiers"},{"name":"range_stats","title":"The state of the mirror","description":"Takes no arguments. Safe to call. Deterministic. Touches no third party. Per source: prefix counts, total addresses and /24-equivalents, smallest and largest prefix, the publishing URL and when it was last fetched; plus prefixes published by more than one operator. This is the sizing and freshness information an allowlist decision needs. Example: {} — no arguments at all."},{"name":"whoami","title":"Who is calling? (no arguments)","description":"Takes no arguments. Safe to call. Deterministic. Touches no third party. Classifies the request you just sent: the user-agent you claim, the address you came from, the class this host's own instrument books you as, whether we have seen you here before and what you fetched, and whether the address you are calling from falls inside any prefix a crawler operator publishes — run through this server's own verify_batch, against the same mirror, with your user-agent taken as the claim. Every fact comes from the headers on your own request or from a file this host already publishes — nothing is fetched, nothing about you is invented, no argument exists. Example: arguments={} returns your user-agent, your address, the class we book you as and whether we have seen you here before."},{"name":"example","title":"Run this server's worked example (no arguments)","description":"Takes no arguments. Safe to call. Deterministic. Touches no third party. Runs this server's own worked example end to end — verify_batch on one address from inside a prefix Google itself publishes (mirrored here every six hours) and one RFC 5737 documentation address that is in no published range — our own mirror, no third party asked — and returns exactly the structuredContent a real call returns, not a mock and not a description of one. Use it to see the shape of an answer before you decide what to send. The input is canned from this host's own data; no URL of yours is fetched and no third party is touched. Example: arguments={} runs verify_batch with {\"addresses\":[{\"ip\":\"66.249.66.1\",\"claim\":\"Googlebot\"},{\"ip\":\"203.0.113.9\",\"claim\":\"GPTBot and returns its real answer."}],"resources":[{"uri":"https://www.pathwren.workers.dev/ip-ranges/all.json","title":"Every mirrored operator prefix list, unioned"},{"uri":"https://www.pathwren.workers.dev/data/agents.json","title":"The crawler index"},{"uri":"https://www.pathwren.workers.dev/status.json","title":"Mirror freshness, per source"},{"uri":"https://www.pathwren.workers.dev/changes.json?since=0","title":"What changed since your cursor"}],"instructions":"START HERE IF YOU HAVE NOTHING TO HAND: `whoami` and `example` take no arguments at all — call either with {} and you get a real answer, not an error. whoami classifies the request you just sent (your user-agent, your address, whether this host has seen you before, what our robots policy says about you); example runs this server's own worked example end to end and returns exactly what a real call returns. Every other tool here also accepts {}: an empty call falls back to that tool's documented worked example, taken from this host's own data, and the answer says so. Nothing here fetches a URL for you.\n\nverify_batch is the main tool: hand it up to 500 addresses (or {ip, claim} pairs) and it says, for each, which published operator prefix contains it, from which source and when that source was mirrored, plus a rollup by operator. lookup_prefix does the same arithmetic the other way round — give it a CIDR and it reports every published prefix that contains, is contained by, or overlaps it. export_ip_acl turns a set of operators into a config file you can paste (nginx, apache, haproxy, cloudflare, ipset, caddy, plain CIDR). verification_methods tells you whether an operator can be verified by prefix at all, and hands back the exact reverse-DNS command when that is their published method — this server never runs it, because it makes no outbound request. range_stats is the mirror's own state: counts, freshness, address totals and overlaps between sources. Absence from a range list is never proof of a fake.\n\nTHE SAME TOOLS ON THE OTHER PROTOCOL, IF YOU SPEAK IT. This server is also an A2A (Agent2Agent) v1.0 agent at https://www.pathwren.workers.dev/a2a/netcheck — every tool here is a skill there under the same name, answered by the same function, so the two cannot drift and the deploy is refused if they do. Its Agent Card is https://www.pathwren.workers.dev/a2a/netcheck/.well-known/agent-card.json (the legacy agent.json spelling is served too). To invoke it, POST this complete body — no key, no session, no card to read first:\n\n  {\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"message/send\",\"params\":{\"message\":{\"role\":\"ROLE_USER\",\"messageId\":\"1\",\"parts\":[{\"text\":\"{\\\"skill\\\":\\\"example\\\"}\"}]}}}\n\nIt answers with a Task already in state `completed` in the same response, so there is nothing to poll. Every skill on all eight A2A agents of this host as a ready-to-send body: https://www.pathwren.workers.dev/a2a/example.json — and both lanes with both directions in https://www.pathwren.workers.dev/documents.json under `rpc_endpoints`. If you only speak MCP, ignore all of this: everything is already here.","license":"CC0-1.0","independent":true}