The Wall

Every monitored site × AI crawlers and search engines, as of 2026-06-20 — 6 of 116 sites block at least one AI crawler by name.

All cohorts Top 1000 websites US 500 Nasdaq 100 AI labs Cybersecurity SEO & AI search Creative rights-holders Knowledge & UGC National indices Banks & fintech European companies Social Networks Governments Publishers Show all bots

blocked by name restricted by name via wildcard only allowed AI crawlers Search engines

Site	GPTBot	ChatGPT-User	ClaudeBot	anthropic-ai	CCBot	Google-Extended	Applebot-Extended	PerplexityBot	Bytespider	Amazonbot	meta-externalagent	FacebookBot	Googlebot	Bingbot	DuckDuckBot	Baiduspider	YandexBot	Applebot
🇯🇵 Toyota
🇯🇵 Nintendo
🇦🇷 Mercado Libre
🇦🇺 Telstra
🇨🇳 Alibaba Group
🇫🇷 Airbus
🇩🇰 A.P. Moller-Maersk
🇩🇪 Adidas
🇩🇪 Allianz
🇧🇷 Ambev
🇿🇦 Anglo American
🇬🇧 Arm Holdings
🇳🇱 ASML
🇬🇧 AstraZeneca
🇫🇷 AXA
🇨🇳 Baidu
🇪🇸 Banco Santander
🇮🇩 Bank Central Asia
🇬🇧 Barclays
🇩🇪 BASF
🇩🇪 Bayer
🇮🇳 Bharti Airtel
🇩🇪 BMW
🇫🇷 BNP Paribas
🇬🇧 BP
🇬🇧 British American Tobacco
🇨🇦 Brookfield
🇨🇦 Canadian National Railway
🇲🇽 Cemex
🇮🇱 Check Point
🇦🇺 Commonwealth Bank
🇦🇺 CSL
🇫🇷 Danone
🇸🇬 DBS Group
🇩🇪 Deutsche Telekom
🇬🇧 Diageo
🇳🇴 DNB
🇨🇴 Ecopetrol
🇨🇦 Enbridge
🇮🇹 Enel
🇮🇹 Eni
🇸🇪 Ericsson
🇲🇽 FEMSA
🇮🇹 Ferrari
🇳🇿 Fonterra
🇵🇹 Galp
🇸🇬 Grab
🇲🇽 Grupo Bimbo
🇬🇧 GSK
🇸🇪 H&M
🇳🇱 Heineken
🇫🇷 Hermes
🇯🇵 Hitachi
🇹🇼 Hon Hai (Foxconn)
🇬🇧 HSBC
🇰🇷 Hyundai Motor
🇪🇸 Iberdrola
🇮🇳 ICICI Bank
🇮🇳 Infosys
🇳🇱 ING Group
🇮🇹 Intesa Sanpaolo
🇧🇷 Itaú Unibanco
🇯🇵 Keyence
🇯🇵 KIOXIA
🇹🇷 Koç Holding
🇨🇳 Kweichow Moutai
🇫🇷 L'Oreal
🇰🇷 LG Corp
🇫🇷 LVMH
🇹🇼 MediaTek
🇩🇪 Mercedes-Benz Group
🇿🇦 MTN Group
🇿🇦 Naspers
🇰🇷 NAVER
🇨🇭 Nestle
🇫🇮 Nokia
🇨🇭 Novartis
🇩🇰 Novo Nordisk
🇦🇹 OMV
🇧🇷 Petrobras
🇲🇾 Petronas
🇵🇱 PKO Bank Polski
🇹🇭 PTT
🇬🇧 Rio Tinto
🇨🇭 Roche
🇨🇦 Royal Bank of Canada
🇰🇷 Samsung Electronics
🇫🇷 Sanofi
🇩🇪 SAP
🇸🇦 Saudi Aramco
🇫🇷 Schneider Electric
🇸🇬 Sea Limited
🇬🇧 Shell
🇨🇦 Shopify
🇩🇪 Siemens
🇰🇷 SK Hynix
🇯🇵 SoftBank Group
🇯🇵 Sony
🇸🇪 Spotify
🇮🇳 Tata Consultancy Services
🇪🇸 Telefonica
🇮🇩 Telkom Indonesia
🇨🇳 Tencent
🇮🇱 Teva Pharmaceutical
🇨🇦 Toronto-Dominion Bank
🇫🇷 TotalEnergies
🇹🇼 TSMC
🇨🇭 UBS
🇬🇧 Unilever
🇧🇷 Vale
🇩🇪 Volkswagen Group
🇸🇪 Volvo Group
🇦🇺 Westpac
🇨🇳 Xiaomi
🇦🇷 YPF
🇨🇳 Zhongji Innolight