All sites · Technology & Software
25 sites in Technology & Software blocking ≥1 AI crawler by name.
All sites
Top 1000 websites
US 500
Nasdaq 100
AI models
Cybersecurity
SEO & AI search
Creative rights-holders
Knowledge & UGC
Academic & open research
Universities
National indices
Banks & fintech
E-commerce
European companies
Social Networks
Governments
Publishers
Sector: Technology & Software · AI blockers ×
| # | Company | Domain | Sector | robots.txt | AI bots blocked | Snapshots | Changes | Last change | |
|---|---|---|---|---|---|---|---|---|---|
| 5 | 🇺🇸 | ChatGPT | chatgpt.com | LLM | present | 🤖 8 | 18 | 17 | 4d ago |
| 11 | 🇺🇸 | Midjourney | midjourney.com | LLM | present | 🤖 13 | 1 | 0 | — |
| 16 | 🇺🇸 | Together AI | together.ai | LLM | present | 🤖 1 | 1 | 0 | — |
| 16 | 🇺🇸 | Proofpoint | proofpoint.com | Cybersecurity | present | 🤖 4 | 31 | 30 | 17h ago |
| 19 | 🇺🇸 | Wired | wired.com | Technology | present | 🤖 20 | 7 | 6 | 15d ago |
| 20 | 🇺🇸 | The Verge | theverge.com | Technology | present | 🤖 23 | 2 | 1 | 1mo ago |
| 21 | 🇺🇸 | TechCrunch | techcrunch.com | Technology | present | 🤖 13 | 1 | 0 | — |
| 22 | 🇺🇸 | Ars Technica | arstechnica.com | Technology | present | 🤖 23 | 1 | 0 | — |
| 27 | 🇺🇸 | Yahoo Search | yahoo.com | Search engine | present | 🤖 14 | 2 | 1 | 1mo ago |
| 38 | 🇺🇸 | Notion | notion.so | Programming and Developer Software | present | 🤖 1 | 1 | 0 | — |
| 39 | 🇺🇸 | Figma Community | figma.com | Graphics Multimedia and Web Design | present | 🤖 13 | 138 | 137 | 17h ago |
| 39 | 🇬🇧 | Sophos | sophos.com | Cybersecurity | present | 🤖 1 | 2 | 1 | 17d ago |
| 40 | 🇺🇸 | Behance | behance.net | Graphics Multimedia and Web Design | present | 🤖 15 | 1 | 0 | — |
| 52 | 🇺🇸 | Arkose Labs | arkoselabs.com | Cybersecurity | present | 🤖 1 | 5 | 4 | 1mo ago |
| 52 | 🇺🇸 | Claude | claude.ai | LLM | present | 🤖 4 | 3 | 2 | 2mo ago |
| 53 | 🇺🇸 | Grok | grok.com | LLM | present | 🤖 6 | 1 | 0 | — |
| 124 | 🇺🇸 | Arrow Electronics | arrow.com | Technology | present | 🤖 6 | 12 | 19 | 10d ago |
| 205 | 🇺🇸 | Airbnb | airbnb.com | Technology | present | 🤖 4 | 2 | 1 | 1mo ago |
| 215 | 🇺🇸 | Electronic Arts | ea.com | Technology | present | 🤖 4 | 2 | 1 | 2mo ago |
| 219 | 🇺🇸 | ServiceNow | servicenow.com | Technology | present | 🤖 1 | 3 | 2 | 3mo ago |
| 403 | 🇺🇸 | eBay | ebay.com | Technology | present | 🤖 9 | 5 | 4 | 2mo ago |
| 420 | 🇺🇸 | Insight Enterprises | insight.com | Technology | present | 🤖 9 | 4 | 3 | 1mo ago |
| 862 | 🇺🇸 | Fortinet | fortinet.com | Cybersecurity | present | 🤖 1 | 2 | 1 | 2mo ago |
| 887 | 🇺🇸 | Roper Technologies | ropertech.com | Programming and Developer Software | present | 🤖 1 | 1 | 0 | — |
| 980 | 🇺🇸 | GitHub | github.com | Programming and Developer Software | present | 🤖 1 | 8 | 7 | 5h ago |