ClaudeBot
Anthropic · LLM training crawler
105 of 1203 monitored sites (8.7%) block ClaudeBot by name, 2 restrict it.
ClaudeBot blocking over time share of monitored sites · weekly snapshot
Sites blocking
ClaudeBot
2026-06-20
2026-11-07
5 earlier weeks omitted (archive building) · 21 weekly snapshots with reliable coverage
Recent ClaudeBot events
- 2026-11-07 — GitHub: Added Allow: /enterprise/contact/data-residency?* for ClaudeBot; Added Allow: /enterprise/contact?* for ClaudeBot; Added Allow: /features/codespaces?* for ClaudeBot … and 4 more change(s) diff
- 2026-11-06 — Advance Auto Parts: Blocked ClaudeBot (Anthropic) entirely diff
- 2026-11-06 — Advance Auto Parts: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-11-05 — Delta Air Lines: Removed rules for AI crawler ClaudeBot (Anthropic) diff
- 2026-11-05 — First American Financial: Blocked ClaudeBot (Anthropic) entirely diff
- 2026-11-05 — Best Buy: Added Allow: / for Claudebot diff
- 2026-11-04 — Bayer: Blocked ClaudeBot (Anthropic) entirely diff
- 2026-11-04 — Varonis: Added rules for AI crawler ClaudeBot (Anthropic) diff
- 2026-11-03 — Advance Auto Parts: Blocked ClaudeBot (Anthropic) entirely diff
- 2026-11-03 — Advance Auto Parts: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-11-01 — Advance Auto Parts: Blocked ClaudeBot (Anthropic) entirely diff
- 2026-11-01 — Advance Auto Parts: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-10-31 — Advance Auto Parts: Blocked ClaudeBot (Anthropic) entirely diff
- 2026-10-31 — Burlington Stores: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-10-30 — Advance Auto Parts: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-10-30 — Advance Auto Parts: Blocked ClaudeBot (Anthropic) entirely diff
- 2026-10-30 — Pixiv: Added Allow: /info.php for ClaudeBot diff
- 2026-10-30 — Advance Auto Parts: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-10-30 — The Asahi Shimbun: Added Allow: /corporate/ for ClaudeBot diff
- 2026-10-29 — Delta Air Lines: Added rules for AI crawler ClaudeBot (Anthropic) diff
- 2026-10-29 — Frontier Communications: Removed rules for AI crawler ClaudeBot (Anthropic) diff
- 2026-10-29 — Zillow: Added rules for AI crawler ClaudeBot (Anthropic) diff
- 2026-10-29 — Nikkei: Added Allow: /promotion/$ for ClaudeBot; Added Allow: /promotion/about/$ for ClaudeBot; Added Allow: /promotion/campaign/student/startguide/$ for ClaudeBot … and 2 more change(s) diff
- 2026-10-28 — Heineken: Added rules for AI crawler ClaudeBot (Anthropic) diff
- 2026-10-27 — Mastodon: Blocked ClaudeBot (Anthropic) entirely diff
- 2026-10-24 — Advance Auto Parts: Blocked ClaudeBot (Anthropic) entirely diff
- 2026-10-24 — Guyana (Office of the President): Blocked ClaudeBot (Anthropic) entirely diff
- 2026-10-24 — Advance Auto Parts: Unblocked ClaudeBot (Anthropic) - rules removed diff
- 2026-10-23 — Advance Auto Parts: Blocked ClaudeBot (Anthropic) entirely diff
- 2026-10-23 — Advance Auto Parts: Unblocked ClaudeBot (Anthropic) - rules removed diff
Sites blocking ClaudeBot
| Site | Sector | Badge |
|---|---|---|
| Advance Auto Parts | Retailing | |
| Airbnb | Technology | |
| Al Jazeera | Broadcast News | |
| AliExpress | E-commerce | |
| Amazon | Retailing | |
| ArcelorMittal | Steel | |
| Arrow Electronics | Technology | |
| Ars Technica | Technology | |
| Bandcamp | Music | |
| Bayer | Pharmaceuticals | |
| BBC | Broadcast News | |
| Behance | Graphics Multimedia and Web Design | |
| bioRxiv | Preprint server | |
| Bloomberg | News & Financial | |
| Business Insider | Business | |
| BuzzFeed | Digital Media | |
| Canva | Web | |
| Casey's General Stores | Retailing | |
| China Mobile | Telecommunications | |
| CNN | Broadcast News | |
| Corriere della Sera | News | |
| Costa Rica (Presidencia) | Americas | |
| Daily Mail | News | |
| De Telegraaf | News | |
| Der Spiegel | News | |
| Die Zeit | News | |
| eBay | Technology | |
| El País | News | |
| Europe PMC | Research repository | |
| Expeditors International | Transportation | |
| Social Network | ||
| Figma Community | Graphics Multimedia and Web Design | |
| Financial Times | News & Financial | |
| First American Financial | Financials | |
| Forbes | Business | |
| Genius | Music | |
| Grok | LLM | |
| Guyana (Office of the President) | Americas | |
| HuffPost | Digital Media | |
| Iceland (Government.is) | Europe | |
| IGN | Web | |
| Insight Enterprises | Technology | |
| Social Network | ||
| Ireland (Gov.ie) | Europe | |
| Kimberly-Clark | Consumer Goods | |
| Le Figaro | News | |
| Le Monde | News | |
| Letterboxd | Arts and Entertainment | |
| Professional Network | ||
| Lithia Motors | Retailing | |
| Los Angeles Times | News | |
| Lufthansa Group | Airlines | |
| Mastodon | Social Network (Fediverse) | |
| Medium | Publishing | |
| Mercado Libre | E-commerce | |
| Msn | Web | |
| NBC News | Broadcast News | |
| Newsweek | Magazine | |
| Nikkei | News & Financial | |
| Nintendo | Video Games | |
| Northrop Grumman | Aerospace & Defense | |
| NPR | Public Radio | |
| Pixiv | Art | |
| Politico | News | |
| Quora | Q&A | |
| Reliance Steel & Aluminum | Materials | |
| Scribd | Public Records and Directories | |
| Shutterstock | Stock media | |
| Sky News | Broadcast News | |
| Slate | Digital Media | |
| SlideShare | Public Records and Directories | |
| Snapchat | Messaging | |
| Sony Music | Music | |
| SoundCloud | Music | |
| Storyblocks | Stock media | |
| TechCrunch | Technology | |
| The Asahi Shimbun | News | |
| The Associated Press | Newswire | |
| The Atlantic | Magazine | |
| The Economist | Magazine | |
| The Globe and Mail | News | |
| The Guardian | News | |
| The Hill | Politics | |
| The New York Times | News | |
| The New Yorker | Magazine | |
| The Sydney Morning Herald | News | |
| The Telegraph | News | |
| The Verge | Technology | |
| The Washington Post | News | |
| Threads | Social Network | |
| Tidal | Music | |
| TikTok | Video | |
| Toronto Star | News | |
| Tripadvisor | Web | |
| Tumblr | Social Network | |
| Uol | Web | |
| USA Today | News | |
| Vimeo | Video | |
| Vox | News | |
| W.W. Grainger | Wholesale | |
| Messaging | ||
| wikiHow | Dictionaries and Encyclopedias | |
| Wired | Technology | |
| Yahoo Search | Search engine | |
| Yelp | Web |