The Wall

Every monitored site × AI crawlers and search engines, as of 2026-06-20 — 7 of 92 sites block at least one AI crawler by name.

All cohorts Top 1000 websites US 500 Nasdaq 100 AI labs Cybersecurity SEO & AI search Creative rights-holders Knowledge & UGC National indices Banks & fintech European companies Social Networks Governments Publishers Show fewer bots

blocked by name restricted by name via wildcard only allowed AI crawlers Search engines

Site	GPTBot	ChatGPT-User	OAI-SearchBot	ClaudeBot	Claude-User	Claude-SearchBot	anthropic-ai	Claude-Web	CCBot	Google-Extended	Applebot-Extended	PerplexityBot	Perplexity-User	Bytespider	Amazonbot	FacebookBot	meta-externalagent	meta-externalfetcher	cohere-ai	AI2Bot	Diffbot	omgili	YouBot	DuckAssistBot	MistralAI-User	PanguBot	Timpibot	Googlebot	Googlebot-Image	Googlebot-News	Bingbot	Slurp	DuckDuckBot	Baiduspider	YandexBot	Applebot	Sogou	SeznamBot	PetalBot
🇺🇸 Amazon
🇺🇸 eBay
🇺🇸 Electronic Arts
🇺🇸 Diamondback Energy
🇺🇸 Fortinet
🇺🇸 Lululemon Athletica
🇺🇸 Roper Technologies
🇺🇸 Adobe
🇺🇸 ADP
🇺🇸 Advanced Micro Devices
🇺🇸 Airbnb
🇺🇸 Align Technology
🇺🇸 Alphabet
🇺🇸 Amgen
🇺🇸 Analog Devices
🇺🇸 Ansys
🇺🇸 Apple
🇺🇸 Applied Materials
🇬🇧 Arm Holdings
🇺🇸 Atlassian
🇺🇸 Autodesk
🇺🇸 Biogen
🇺🇸 Booking Holdings
🇺🇸 Broadcom
🇺🇸 Charter Communications
🇺🇸 Cintas
🇺🇸 Cisco Systems
🇺🇸 Cognizant
🇺🇸 Coinbase
🇺🇸 Comcast
🇺🇸 Constellation Energy
🇺🇸 Copart
🇺🇸 Costco Wholesale
🇺🇸 CrowdStrike
🇺🇸 Datadog
🇺🇸 DexCom
🇺🇸 Dollar Tree
🇺🇸 DoorDash
🇺🇸 Enphase Energy
🇺🇸 Exelon
🇺🇸 Expedia Group
🇺🇸 Fastenal
🇺🇸 First Solar
🇺🇸 GE HealthCare
🇺🇸 Gilead Sciences
🇺🇸 GlobalFoundries
🇺🇸 Honeywell International
🇺🇸 IDEXX Laboratories
🇺🇸 Intel
🇺🇸 Intuit
🇺🇸 Intuitive Surgical
🇺🇸 Keurig Dr Pepper
🇺🇸 KLA
🇺🇸 Lam Research
🇺🇸 Linde
🇺🇸 Marriott International
🇺🇸 Marvell Technology
🇺🇸 Meta Platforms
🇺🇸 Microchip Technology
🇺🇸 Micron Technology
🇺🇸 Microsoft
🇺🇸 Mondelez International
🇺🇸 MongoDB
🇺🇸 Netflix
🇺🇸 Nvidia
🇺🇸 O'Reilly Automotive
🇺🇸 Okta
🇺🇸 Old Dominion Freight Line
🇺🇸 Onsemi
🇺🇸 Palantir
🇺🇸 Palo Alto Networks
🇺🇸 PayPal Holdings
🇺🇸 PepsiCo
🇺🇸 Qualcomm
🇺🇸 Regeneron Pharmaceuticals
🇺🇸 Ross Stores
🇺🇸 Sirius XM
🇺🇸 Snowflake
🇺🇸 Starbucks
🇺🇸 Super Micro Computer
🇺🇸 Synopsys
🇺🇸 Take-Two Interactive
🇺🇸 Tesla
🇺🇸 Texas Instruments
🇺🇸 The Trade Desk
🇺🇸 Verisign
🇺🇸 Vertex Pharmaceuticals
🇺🇸 Walgreens Boots Alliance
🇺🇸 Warner Bros. Discovery
🇺🇸 Workday
🇺🇸 Zoom Communications
🇺🇸 Zscaler