The Wall

Every monitored site × AI crawlers and search engines, as of 2026-06-20 — 1 of 46 sites block at least one AI crawler by name.

All cohorts Top 1000 websites US 500 Nasdaq 100 AI labs Cybersecurity SEO & AI search Creative rights-holders Knowledge & UGC National indices Banks & fintech European companies Social Networks Governments Publishers Show fewer bots

blocked by name restricted by name via wildcard only allowed AI crawlers Search engines

Site	GPTBot	ChatGPT-User	OAI-SearchBot	ClaudeBot	Claude-User	Claude-SearchBot	anthropic-ai	Claude-Web	CCBot	Google-Extended	Applebot-Extended	PerplexityBot	Perplexity-User	Bytespider	Amazonbot	FacebookBot	meta-externalagent	meta-externalfetcher	cohere-ai	AI2Bot	Diffbot	omgili	YouBot	DuckAssistBot	MistralAI-User	PanguBot	Timpibot	Googlebot	Googlebot-Image	Googlebot-News	Bingbot	Slurp	DuckDuckBot	Baiduspider	YandexBot	Applebot	Sogou	SeznamBot	PetalBot
🇫🇷 Qonto
🇳🇱 Adyen
🇺🇸 Affirm
🇺🇸 American Express
🇦🇺 ANZ
🇪🇸 Banco Santander
🇺🇸 Bank of America
🇬🇧 Barclays
🇺🇸 Block
🇫🇷 BNP Paribas
🇺🇸 Brex
🇺🇸 Capital One Financial
🇺🇸 Chime
🇺🇸 Citigroup
🇺🇸 Coinbase
🇦🇺 Commonwealth Bank
🇫🇷 Credit Agricole
🇩🇪 Deutsche Bank
🇺🇸 Goldman Sachs
🇬🇧 HSBC
🇳🇱 ING Group
🇺🇸 JPMorgan Chase
🇸🇪 Klarna
🇺🇸 Kraken
🇺🇸 Mastercard
🇺🇸 Mercury
🇬🇧 Monzo
🇺🇸 Morgan Stanley
🇩🇪 N26
🇧🇷 Nubank
🇺🇸 PayPal Holdings
🇺🇸 Plaid
🇺🇸 Ramp
🇬🇧 Revolut
🇺🇸 Robinhood
🇨🇦 Royal Bank of Canada
🇺🇸 SoFi
🇬🇧 Standard Chartered
🇬🇧 Starling Bank
🇺🇸 Stripe
🇨🇦 Toronto-Dominion Bank
🇨🇭 UBS
🇺🇸 Visa
🇨🇦 Wealthsimple
🇺🇸 Wells Fargo
🇬🇧 Wise