All sites · News & Media
44 sites in News & Media blocking ≥1 AI crawler by name.
All sites
Top 1000 websites
US 500
Nasdaq 100
AI models
Cybersecurity
SEO & AI search
Creative rights-holders
Knowledge & UGC
Academic & open research
Universities
National indices
Banks & fintech
E-commerce
European companies
Social Networks
Governments
Publishers
Sector: News & Media · AI blockers ×
| # | Company | Domain | Sector | robots.txt | AI bots blocked | Snapshots | Changes | Last change | |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 🇺🇸 | The New York Times | nytimes.com | News | present | 🤖 24 | 6 | 5 | 24d ago |
| 2 | 🇺🇸 | The Washington Post | washingtonpost.com | News | present | 🤖 13 | 4 | 3 | 2mo ago |
| 4 | 🇺🇸 | USA Today | usatoday.com | News | present | 🤖 28 | 9 | 8 | 1mo ago |
| 5 | 🇺🇸 | Los Angeles Times | latimes.com | News | present | 🤖 5 | 1 | 0 | — |
| 7 | 🇺🇸 | Reuters | reuters.com | Newswire | present | 🤖 1 | 13 | 12 | 9d ago |
| 8 | 🇺🇸 | The Associated Press | apnews.com | Newswire | present | 🤖 13 | 8 | 7 | 2d ago |
| 9 | 🇺🇸 | CNN | cnn.com | Broadcast News | present | 🤖 24 | 8 | 7 | 15d ago |
| 11 | 🇺🇸 | NBC News | nbcnews.com | Broadcast News | present | 🤖 24 | 6 | 5 | 15d ago |
| 12 | 🇺🇸 | CBS News | cbsnews.com | Broadcast News | present | 🤖 1 | 1 | 0 | — |
| 13 | 🇺🇸 | ABC News | abcnews.go.com | Broadcast News | present | 🤖 8 | 2 | 1 | 2mo ago |
| 15 | 🇺🇸 | Politico | politico.com | News | present | 🤖 27 | 1 | 0 | — |
| 16 | 🇺🇸 | Medium | medium.com | Publishing | present | 🤖 7 | 1 | 0 | — |
| 16 | 🇺🇸 | Axios | axios.com | News | present | 🤖 5 | 2 | 1 | 4mo ago |
| 17 | 🇺🇸 | The Atlantic | theatlantic.com | Magazine | present | 🤖 19 | 4 | 3 | 1d ago |
| 18 | 🇺🇸 | The New Yorker | newyorker.com | Magazine | present | 🤖 20 | 6 | 5 | 1mo ago |
| 23 | 🇺🇸 | Vox | vox.com | News | present | 🤖 23 | 4 | 3 | 2mo ago |
| 24 | 🇺🇸 | BuzzFeed | buzzfeed.com | Digital Media | present | 🤖 18 | 2 | 1 | 4mo ago |
| 28 | 🇺🇸 | Newsweek | newsweek.com | Magazine | present | 🤖 14 | 3 | 2 | 8d ago |
| 31 | 🇺🇸 | HuffPost | huffpost.com | Digital Media | present | 🤖 22 | 1 | 0 | — |
| 32 | 🇺🇸 | Slate | slate.com | Digital Media | present | 🤖 15 | 11 | 10 | 4d ago |
| 38 | 🇬🇧 | BBC | bbc.com | Broadcast News | present | 🤖 20 | 8 | 7 | 5d ago |
| 39 | 🇬🇧 | The Guardian | theguardian.com | News | present | 🤖 13 | 2 | 1 | 4mo ago |
| 42 | 🇬🇧 | The Telegraph | telegraph.co.uk | News | present | 🤖 25 | 8 | 7 | 1d ago |
| 44 | 🇬🇧 | Daily Mail | dailymail.co.uk | News | present | 🤖 28 | 8 | 7 | 2d ago |
| 45 | 🇬🇧 | The Economist | economist.com | Magazine | present | 🤖 14 | 6 | 5 | 1mo ago |
| 46 | 🇬🇧 | Sky News | news.sky.com | Broadcast News | present | 🤖 16 | 1 | 0 | — |
| 47 | 🇫🇷 | Le Monde | lemonde.fr | News | present | 🤖 21 | 4 | 3 | 1d ago |
| 48 | 🇫🇷 | Le Figaro | lefigaro.fr | News | present | 🤖 16 | 6 | 5 | 1mo ago |
| 50 | 🇩🇪 | Der Spiegel | spiegel.de | News | present | 🤖 21 | 5 | 4 | 25d ago |
| 51 | 🇩🇪 | Die Zeit | zeit.de | News | present | 🤖 14 | 5 | 4 | 1mo ago |
| 52 | 🇩🇪 | Bild | bild.de | News | present | 🤖 15 | 7 | 6 | 23h ago |
| 53 | 🇪🇸 | El PaÃs | elpais.com | News | present | 🤖 10 | 4 | 3 | 1mo ago |
| 54 | 🇮🇹 | Corriere della Sera | corriere.it | News | present | 🤖 25 | 16 | 15 | 3d ago |
| 55 | 🇳🇱 | De Telegraaf | telegraaf.nl | News | present | 🤖 23 | 8 | 7 | 10d ago |
| 57 | 🇯🇵 | The Asahi Shimbun | asahi.com | News | present | 🤖 23 | 3 | 2 | 8d ago |
| 59 | 🇨🇦 | The Globe and Mail | theglobeandmail.com | News | present | 🤖 26 | 1 | 0 | — |
| 60 | 🇨🇦 | Toronto Star | thestar.com | News | present | 🤖 28 | 523 | 522 | 5h ago |
| 61 | 🇦🇺 | The Sydney Morning Herald | smh.com.au | News | present | 🤖 21 | 1 | 0 | — |
| 62 | 🇮🇳 | The Times of India | timesofindia.indiatimes.com | News | present | 🤖 1 | 2 | 1 | 3mo ago |
| 63 | 🇶🇦 | Al Jazeera | aljazeera.com | Broadcast News | present | 🤖 8 | 4 | 3 | 8d ago |
| 579 | 🇺🇸 | Shutterstock | shutterstock.com | Stock media | present | 🤖 4 | 2 | 1 | 3mo ago |
| 580 | 🇺🇸 | Adobe Stock | stock.adobe.com | Stock media | present | 🤖 6 | 1 | 0 | — |
| 582 | 🇺🇸 | Pond5 | pond5.com | Stock media | present | 🤖 3 | 1 | 0 | — |
| 583 | 🇺🇸 | Storyblocks | storyblocks.com | Stock media | present | 🤖 10 | 1 | 0 | — |