AI2Bot (Allen Institute for AI)

The Allen Institute's crawler, gathering pages for open research corpora such as Dolma that underpin fully open models like OLMo. Blocking it costs you: Excluded from open research datasets. Worth a deliberate decision: this is the category where 'blocking AI' also blocks the open, auditable end of it. robots.txt token: AI2Bot.