CCBot

Recent CCBot events

• 2026-09-18 — Commonwealth Bank: Added Allow: /digital/home-loans/borrowing-power-calculator for CCBot; Added Allow: /etc/designs/commbank/favicon.ico for CCBot diff
• 2026-09-16 — Sony Music: Unblocked CCBot (Common Crawl) - rules removed diff
• 2026-09-16 — Expeditors International: Unblocked CCBot (Common Crawl) - rules removed diff
• 2026-09-16 — Tenet Healthcare: Unblocked CCBot (Common Crawl) - rules removed diff
• 2026-09-16 — Guyana (Office of the President): Unblocked CCBot (Common Crawl) - rules removed diff
• 2026-09-16 — Costa Rica (Presidencia): Unblocked CCBot (Common Crawl) - rules removed diff
• 2026-09-16 — EPFL: Unblocked CCBot (Common Crawl) - rules removed diff
• 2026-09-16 — AI21 Labs: Unblocked CCBot (Common Crawl) - rules removed diff
• 2026-09-16 — Iceland (Government.is): Unblocked CCBot (Common Crawl) - rules removed diff
• 2026-09-11 — Canva: Added Allow: /*/ai-connector/ for CCBot; Added Allow: /ai-connector/ for CCBot; Added Allow: /developers/ for CCBot diff
• 2026-09-11 — Coursera: Added Allow: /courses?*&page=1$ for CCBot; Added Allow: /courses?*&page=1& for CCBot; Added Allow: /courses?*&page=2$ for CCBot … and 19 more change(s) diff
• 2026-09-09 — Commonwealth Bank: Removed Disallow: /personal/eform/ for CCBot diff
• 2026-09-07 — DBLP: Blocked CCBot (Common Crawl) entirely diff
• 2026-09-04 — Canva: Added Allow: /*/design-school/ for CCBot; Added Allow: /design-school/ for CCBot diff
• 2026-08-31 — Roblox: Unblocked CCBot (Common Crawl) - rules removed diff
• 2026-08-26 — IGN: Added Allow: /sponsored/ for CCBot diff
• 2026-08-25 — Qwant: Added Disallow: /?*bottomMenu=* for CCBot; Added Disallow: /?*drawer=* for CCBot; Added Disallow: /?*home=* for CCBot … and 1 more change(s) diff
• 2026-08-25 — Europe PMC: Blocked CCBot (Common Crawl) entirely diff
• 2026-08-24 — Best Buy: Added rules for AI crawler CCBot (Common Crawl) diff
• 2026-08-22 — Barracuda: Unblocked CCBot (Common Crawl) - rules removed diff
• 2026-08-20 — Vox: Added Allow: /strategist for CCBot diff
• 2026-08-19 — Vox: Removed Allow: /strategist for CCBot diff
• 2026-08-19 — Vox: Added Allow: /strategist for CCBot diff
• 2026-08-18 — Reliance Steel & Aluminum: Blocked CCBot (Common Crawl) entirely diff
• 2026-08-18 — DOAJ: Unblocked CCBot (Common Crawl) - rules removed diff
• 2026-08-17 — AstraZeneca: Added rules for AI crawler CCBot (Common Crawl) diff
• 2026-08-17 — Itaú Unibanco: Added rules for AI crawler CCBot (Common Crawl) diff
• 2026-08-14 — KeyCorp: Blocked CCBot (Common Crawl) entirely diff
• 2026-08-12 — Nvidia: Removed rules for AI crawler CCBot (Common Crawl) diff
• 2026-08-11 — Roblox: Blocked CCBot (Common Crawl) entirely diff

Sites blocking CCBot

Site	Sector	Badge
ABC News	Broadcast News
Adobe Stock	Stock media
Amazon	Retailing
ArcelorMittal	Steel
Archive of Our Own	Books
Arrow Electronics	Technology
Ars Technica	Technology
Axios	News
Bandcamp	Music
Baxter International	Health Care
BBC	Broadcast News
Behance	Graphics Multimedia and Web Design
Bild	News
bioRxiv	Preprint server
Bloomberg	News & Financial
Burlington Stores	Retailing
Business Insider	Business
BuzzFeed	Digital Media
Canva	Web
Casey's General Stores	Retailing
ChatGPT	LLM
China Mobile	Telecommunications
CNN	Broadcast News
Corriere della Sera	News
Daily Mail	News
De Telegraaf	News
Der Spiegel	News
Die Zeit	News
eBay	Technology
El País	News
ESPN	Web
Europe PMC	Research repository
Figma Community	Graphics Multimedia and Web Design
Financial Times	News & Financial
Forbes	Business
Genius	Music
Globo	Web
Goodreads	Books
HuffPost	Digital Media
IGN	Web
Insight Enterprises	Technology
KeyCorp	Financials
Kimberly-Clark	Consumer Goods
Le Figaro	News
Le Monde	News
Letterboxd	Arts and Entertainment
LinkedIn	Professional Network
Lithia Motors	Retailing
Midjourney	LLM
Msn	Web
NBC News	Broadcast News
Newsweek	Magazine
Nikkei	News & Financial
Nintendo	Video Games
Northrop Grumman	Aerospace & Defense
NPR	Public Radio
Parker Hannifin	Industrials
Pixiv	Art
Politico	News
Pond5	Stock media
Prudential Financial	Financials
Reliance Steel & Aluminum	Materials
Scribd	Public Records and Directories
Sempra	Energy
ServiceNow	Technology
Shutterstock	Stock media
Sky News	Broadcast News
Slate	Digital Media
SlideShare	Public Records and Directories
Snapchat	Messaging
SoundCloud	Music
Storyblocks	Stock media
TechCrunch	Technology
Telstra	Telecommunications
The Asahi Shimbun	News
The Associated Press	Newswire
The Atlantic	Magazine
The Economist	Magazine
The Globe and Mail	News
The Guardian	News
The Hill	Politics
The Intercept	Investigative
The New York Times	News
The New Yorker	Magazine
The Sydney Morning Herald	News
The Telegraph	News
The Verge	Technology
The Washington Post	News
Tidal	Music
TikTok	Video
Toronto Star	News
Toyota	Motor Vehicles
Tripadvisor	Web
Tumblr	Social Network
Uol	Web
USA Today	News
Vimeo	Video
Vox	News
Weather	Weather
wikiHow	Dictionaries and Encyclopedias
Wired	Technology
WPP	Advertising
Yahoo Search	Search engine
Yelp	Web

Sites restricting CCBot

Itaú Unibanco