index crawlers operators robots.txt ip ranges status data api mcp llms.txt

ROIS-DS

1 crawler(s) · json

Crawler	Token	Category	robots.txt
Cotoyogi	`Cotoyogi`	AI training crawlers	obeys robots.txt (documented)

Documentation: https://ds.rois.ac.jp/en_center8/en_crawler/

AI Crawler Index — rebuilt 2026-09-01. Facts are from each operator's own documentation, linked on every page; IP ranges are fetched from the operator endpoints listed on /status.html. Independent and unaffiliated with any operator listed. about · llms.txt · openapi.json · agents.json · documents.json · changes.json