Webz.io · Corpus and dataset builders · json
User-agent: omgili Disallow: /
| robots.txt token | omgili |
| User-agent contains | omgili |
| Operator | Webz.io |
| Category | Corpus and dataset builders |
| robots.txt | obeys robots.txt (documented) |
| Verify by | no published verification method |
The older robots token for the same Webz.io collection, still honoured and still worth listing.
Same as omgilibot.
omgili/0.5 +http://omgili.com
User-agent: omgili Allow: /
Operator documentation: https://webz.io/blog/machine-learning/
Machine copies: json ·
markdown
Policies that name this crawler:
allow-all · block-all-ai · block-datasets · maximum-ai-visibility