The Wall

Every monitored site × AI crawlers and search engines, as of 2025-04-28 — 9 of 36 sites block at least one AI crawler by name.

All cohorts Top 1000 websites US 500 Nasdaq 100 AI models Cybersecurity SEO & AI search Creative rights-holders Knowledge & UGC Academic & open research Universities National indices Banks & fintech E-commerce European companies Social Networks Governments Publishers Show all bots

blocked by name restricted by name via wildcard only allowed AI crawlers Search engines

Site	GPTBot	ChatGPT-User	ClaudeBot	anthropic-ai	CCBot	Google-Extended	Applebot-Extended	PerplexityBot	Bytespider	Amazonbot	meta-externalagent	FacebookBot	Googlebot	Bingbot	DuckDuckBot	Baiduspider	YandexBot	Applebot
🇺🇸 Condé Nast
🇺🇸 Walt Disney
🇩🇪 Axel Springer
🇸🇪 Spotify
🇺🇸 Paramount Global
🇺🇸 Warner Bros. Discovery
🇺🇸 Getty Images
🇺🇸 Shutterstock
🇺🇸 Adobe Stock
🇬🇧 Alamy
🇺🇸 Pond5
🇺🇸 Storyblocks
🇺🇸 Universal Music Group
🇺🇸 Sony Music
🇺🇸 Warner Music Group
🇩🇪 SoundCloud
🇺🇸 Bandcamp
🇫🇷 Believe
🇸🇪 Epidemic Sound
🇺🇸 Penguin Random House
🇺🇸 HarperCollins
🇺🇸 Hachette Book Group
🇺🇸 Simon & Schuster
🇬🇧 Pearson
🇳🇱 Elsevier
🇩🇪 Springer Nature
🇺🇸 Wiley
🇺🇸 NBCUniversal
🇺🇸 Sony Pictures
🇺🇸 Lionsgate
🇺🇸 ArtStation
🇯🇵 Pixiv
🇨🇦 500px
🇬🇧 Magnum Photos
🇺🇸 IMDb
🇺🇸 Discogs