ClaudeBot
Anthropic · LLM training crawler
97 of 1201 monitored sites (8.1%) block ClaudeBot by name, 2 restrict it.
ClaudeBot blocking over time share of monitored sites · weekly snapshot
Sites blocking
ClaudeBot
2026-06-21
2026-09-20
12 earlier weeks omitted (archive building) · 14 weekly snapshots with reliable coverage
Recent ClaudeBot events
- 2026-09-18 — Commonwealth Bank: Added Allow: /digital/home-loans/borrowing-power-calculator for ClaudeBot; Added Allow: /etc/designs/commbank/favicon.ico for ClaudeBot diff
- 2026-09-17 — GoDaddy: Added Allow: */domain-auctions/ for ClaudeBot; Added Allow: */domainsearch/ for ClaudeBot; Added Disallow: */domain-auctions/api/* for ClaudeBot … and 2 more change(s) diff
- 2026-09-17 — Best Buy: Removed Disallow: *?cp=*&id=*&st=* for Claudebot; Removed Disallow: *?id*_facet*&st=* for Claudebot; Removed Disallow: *?id*_facet*_facet*&st=* for Claudebot diff
- 2026-09-16 — GoDaddy: Added rules for AI crawler ClaudeBot (Anthropic) diff
- 2026-09-16 — Weather: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-09-16 — Sony Music: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-09-16 — Expeditors International: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-09-16 — Tenet Healthcare: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-09-16 — Guyana (Office of the President): Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-09-16 — Costa Rica (Presidencia): Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-09-16 — EPFL: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-09-16 — AI21 Labs: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-09-16 — Iceland (Government.is): Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-09-15 — xAI: Removed rules for AI crawler ClaudeBot (Anthropic) diff
- 2026-09-14 — El Corte Ingles: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-09-12 — GitHub: Added rules for AI crawler ClaudeBot (Anthropic) diff
- 2026-09-11 — Zscaler: Added Disallow: /8199ac63-be6c-4cba-9bd9-45500a02d64f.txt for ClaudeBot; Removed Disallow: /indexNowKey.txt for ClaudeBot diff
- 2026-09-11 — Canva: Added Allow: /*/ai-connector/ for ClaudeBot; Added Allow: /ai-connector/ for ClaudeBot; Added Allow: /developers/ for ClaudeBot diff
- 2026-09-09 — SAP: Added Disallow: /fetch for ClaudeBot; Added Disallow: /proxy for ClaudeBot; Added Disallow: /sharedlibs/ for ClaudeBot diff
- 2026-09-09 — Commonwealth Bank: Removed Disallow: /personal/eform/ for ClaudeBot diff
- 2026-09-08 — LG Corp: Removed rules for AI crawler ClaudeBot (Anthropic) diff
- 2026-09-07 — Intel: Added rules for AI crawler ClaudeBot (Anthropic) diff
- 2026-09-07 — DBLP: Blocked ClaudeBot (Anthropic) entirely diff
- 2026-09-04 — Canva: Added Allow: /*/design-school/ for ClaudeBot; Added Allow: /design-school/ for ClaudeBot diff
- 2026-09-03 — Best Buy: Removed Disallow: */site/searchpage.jsp?st=* for Claudebot diff
- 2026-08-31 — Roblox: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-08-26 — IGN: Added Allow: /sponsored/ for ClaudeBot diff
- 2026-08-25 — Europe PMC: Blocked ClaudeBot (Anthropic) entirely diff
- 2026-08-24 — Best Buy: Added rules for AI crawler ClaudeBot (Anthropic) diff
- 2026-08-20 — Vox: Added Allow: /strategist for ClaudeBot diff
Sites blocking ClaudeBot
| Site | Sector | Badge |
|---|---|---|
| Al Jazeera | Broadcast News | |
| Amazon | Retailing | |
| ArcelorMittal | Steel | |
| Arrow Electronics | Technology | |
| Ars Technica | Technology | |
| Bandcamp | Music | |
| Baxter International | Health Care | |
| BBC | Broadcast News | |
| Behance | Graphics Multimedia and Web Design | |
| bioRxiv | Preprint server | |
| Bloomberg | News & Financial | |
| Burlington Stores | Retailing | |
| Business Insider | Business | |
| BuzzFeed | Digital Media | |
| Canva | Web | |
| Casey's General Stores | Retailing | |
| China Mobile | Telecommunications | |
| CNN | Broadcast News | |
| Corriere della Sera | News | |
| Daily Mail | News | |
| DBLP | CS bibliography | |
| De Telegraaf | News | |
| Der Spiegel | News | |
| Die Zeit | News | |
| eBay | Technology | |
| El País | News | |
| Europe PMC | Research repository | |
| Figma Community | Graphics Multimedia and Web Design | |
| Financial Times | News & Financial | |
| Forbes | Business | |
| Genius | Music | |
| Grok | LLM | |
| HuffPost | Digital Media | |
| IGN | Web | |
| Social Network | ||
| Ireland (Gov.ie) | Europe | |
| Kimberly-Clark | Consumer Goods | |
| Le Figaro | News | |
| Le Monde | News | |
| Letterboxd | Arts and Entertainment | |
| Professional Network | ||
| Lithia Motors | Retailing | |
| Los Angeles Times | News | |
| Lufthansa Group | Airlines | |
| Medium | Publishing | |
| Mercado Libre | E-commerce | |
| Msn | Web | |
| NBC News | Broadcast News | |
| Newsweek | Magazine | |
| Nikkei | News & Financial | |
| Nintendo | Video Games | |
| Northrop Grumman | Aerospace & Defense | |
| NPR | Public Radio | |
| Pixiv | Art | |
| Politico | News | |
| Quora | Q&A | |
| Reliance Steel & Aluminum | Materials | |
| Scribd | Public Records and Directories | |
| Sempra | Energy | |
| Shutterstock | Stock media | |
| Sky News | Broadcast News | |
| Slate | Digital Media | |
| SlideShare | Public Records and Directories | |
| Snapchat | Messaging | |
| SoundCloud | Music | |
| Storyblocks | Stock media | |
| TechCrunch | Technology | |
| The Asahi Shimbun | News | |
| The Associated Press | Newswire | |
| The Atlantic | Magazine | |
| The Economist | Magazine | |
| The Globe and Mail | News | |
| The Guardian | News | |
| The Hill | Politics | |
| The New York Times | News | |
| The New Yorker | Magazine | |
| The Sydney Morning Herald | News | |
| The Telegraph | News | |
| The Verge | Technology | |
| The Washington Post | News | |
| Threads | Social Network | |
| Tidal | Music | |
| TikTok | Video | |
| Toronto Star | News | |
| Toyota | Motor Vehicles | |
| Tripadvisor | Web | |
| Tumblr | Social Network | |
| Uol | Web | |
| USA Today | News | |
| Vimeo | Video | |
| Vox | News | |
| W.W. Grainger | Wholesale | |
| Messaging | ||
| wikiHow | Dictionaries and Encyclopedias | |
| Wired | Technology | |
| Yahoo Search | Search engine | |
| Yelp | Web |