GPTBot
OpenAI · LLM training crawler
111 of 1203 monitored sites (9.2%) block GPTBot by name, 3 restrict it.
GPTBot blocking over time share of monitored sites · weekly snapshot
Sites blocking
GPTBot
2026-06-22
2026-11-09
5 earlier weeks omitted (archive building) · 21 weekly snapshots with reliable coverage
Recent GPTBot events
- 2026-11-09 — Advance Auto Parts: Blocked GPTBot (OpenAI) entirely diff
- 2026-11-08 — Advance Auto Parts: Unblocked GPTBot (OpenAI) - rules removed diff
- 2026-11-07 — GitHub: Added Allow: /enterprise/contact/data-residency?* for GPTBot; Added Allow: /enterprise/contact?* for GPTBot; Added Allow: /features/codespaces?* for GPTBot … and 4 more change(s) diff
- 2026-11-06 — Advance Auto Parts: Blocked GPTBot (OpenAI) entirely diff
- 2026-11-06 — Advance Auto Parts: Unblocked GPTBot (OpenAI) - rules removed diff
- 2026-11-05 — Delta Air Lines: Removed rules for AI crawler GPTBot (OpenAI) diff
- 2026-11-05 — The Atlantic: Removed Disallow: /4624/TheAtlanticOnline/* for GPTBot diff
- 2026-11-05 — First American Financial: Blocked GPTBot (OpenAI) entirely diff
- 2026-11-05 — Best Buy: Added Allow: / for GPTBot; Removed Disallow: /[1.1] for GPTBot-Web diff
- 2026-11-04 — Samsung Electronics: Added Allow: /sg/multistore/sg_eppfinancing/offers_epp/ for GPTBot; Removed Allow: /sg/multistore/sg_eppfinancing/offers_epp/sc_premium/ for GPTBot diff
- 2026-11-04 — Bayer: Blocked GPTBot (OpenAI) entirely diff
- 2026-11-04 — Varonis: Added rules for AI crawler GPTBot (OpenAI) diff
- 2026-11-03 — Advance Auto Parts: Blocked GPTBot (OpenAI) entirely diff
- 2026-11-03 — Advance Auto Parts: Unblocked GPTBot (OpenAI) - rules removed diff
- 2026-11-01 — Advance Auto Parts: Blocked GPTBot (OpenAI) entirely diff
- 2026-11-01 — Advance Auto Parts: Unblocked GPTBot (OpenAI) - rules removed diff
- 2026-10-31 — Universal Music Group: Blocked GPTBot (OpenAI) entirely diff
- 2026-10-31 — Advance Auto Parts: Blocked GPTBot (OpenAI) entirely diff
- 2026-10-31 — Burlington Stores: Unblocked GPTBot (OpenAI) - rules removed diff
- 2026-10-30 — Universal Music Group: Unblocked GPTBot (OpenAI) - rules removed diff
- 2026-10-30 — Advance Auto Parts: Unblocked GPTBot (OpenAI) - rules removed diff
- 2026-10-30 — Advance Auto Parts: Blocked GPTBot (OpenAI) entirely diff
- 2026-10-30 — Pixiv: Added Allow: /info.php for GPTBot diff
- 2026-10-30 — Advance Auto Parts: Unblocked GPTBot (OpenAI) - rules removed diff
- 2026-10-30 — The Asahi Shimbun: Added Allow: /corporate/ for GPTBot diff
- 2026-10-29 — Delta Air Lines: Added rules for AI crawler GPTBot (OpenAI) diff
- 2026-10-29 — Daily Mail: Blocked user-agent ‘GPTBot-Search’ entirely diff
- 2026-10-29 — Frontier Communications: Removed rules for AI crawler GPTBot (OpenAI) diff
- 2026-10-28 — Heineken: Added rules for AI crawler GPTBot (OpenAI) diff
- 2026-10-27 — Arrow Electronics: Unblocked GPTBot (OpenAI); Added Allow: / for GPTBot; Removed Allow: /ais/* for GPTBot … and 1 more change(s) diff
Sites blocking GPTBot
| Site | Sector | Badge |
|---|---|---|
| ABC News | Broadcast News | |
| Academia.edu | Research network | |
| Adobe Stock | Stock media | |
| Advance Auto Parts | Retailing | |
| Airbnb | Technology | |
| Al Jazeera | Broadcast News | |
| Alibaba Group | E-commerce | |
| AliExpress | E-commerce | |
| Amazon | Retailing | |
| ArcelorMittal | Steel | |
| Archive of Our Own | Books | |
| Bandcamp | Music | |
| Bayer | Pharmaceuticals | |
| BBC | Broadcast News | |
| Behance | Graphics Multimedia and Web Design | |
| bioRxiv | Preprint server | |
| Bloomberg | News & Financial | |
| BuzzFeed | Digital Media | |
| Canva | Web | |
| Casey's General Stores | Retailing | |
| CBS News | Broadcast News | |
| China Mobile | Telecommunications | |
| Claude | LLM | |
| CNN | Broadcast News | |
| Corriere della Sera | News | |
| Costa Rica (Presidencia) | Americas | |
| Daily Mail | News | |
| De Telegraaf | News | |
| Der Spiegel | News | |
| Disney | Web | |
| eBay | Technology | |
| ESPN | Web | |
| Europe PMC | Research repository | |
| Expeditors International | Transportation | |
| Social Network | ||
| Figma Community | Graphics Multimedia and Web Design | |
| First American Financial | Financials | |
| Forbes | Business | |
| Genius | Music | |
| Globo | Web | |
| Goodreads | Books | |
| Grok | LLM | |
| Guyana (Office of the President) | Americas | |
| HuffPost | Digital Media | |
| Iceland (Government.is) | Europe | |
| IGN | Web | |
| Insight Enterprises | Technology | |
| Social Network | ||
| Ireland (Gov.ie) | Europe | |
| Khan Academy | Education | |
| Kimberly-Clark | Consumer Goods | |
| Le Figaro | News | |
| Letterboxd | Arts and Entertainment | |
| Professional Network | ||
| Lithia Motors | Retailing | |
| Los Angeles Times | News | |
| Lufthansa Group | Airlines | |
| Mastodon | Social Network (Fediverse) | |
| Medium | Publishing | |
| Mercado Libre | E-commerce | |
| Midjourney | LLM | |
| Msn | Web | |
| NBC News | Broadcast News | |
| Newsweek | Magazine | |
| Nextdoor | Social Network | |
| Nikkei | News & Financial | |
| Nintendo | Video Games | |
| Northrop Grumman | Aerospace & Defense | |
| NPR | Public Radio | |
| PG&E | Energy | |
| Pixiv | Art | |
| Politico | News | |
| Pond5 | Stock media | |
| Prudential Financial | Financials | |
| Quora | Q&A | |
| Reliance Steel & Aluminum | Materials | |
| Shutterstock | Stock media | |
| Sky News | Broadcast News | |
| Slate | Digital Media | |
| Snapchat | Messaging | |
| Sony Music | Music | |
| SoundCloud | Music | |
| SSRN | Preprint server | |
| Storyblocks | Stock media | |
| TechCrunch | Technology | |
| The Asahi Shimbun | News | |
| The Associated Press | Newswire | |
| The Economist | Magazine | |
| The Globe and Mail | News | |
| The Hill | Politics | |
| The Intercept | Investigative | |
| The New York Times | News | |
| The Sydney Morning Herald | News | |
| The Telegraph | News | |
| Threads | Social Network | |
| Tidal | Music | |
| TikTok | Video | |
| Toronto Star | News | |
| Tripadvisor | Web | |
| Universal Music Group | Music | |
| University of Cape Town | University | |
| USA Today | News | |
| Vimeo | Video | |
| W.W. Grainger | Wholesale | |
| Wattpad | Books | |
| Messaging | ||
| wikiHow | Dictionaries and Encyclopedias | |
| WPP | Advertising | |
| Yahoo | Web | |
| Yahoo Search | Search engine | |
| Yelp | Web |