CNN / robots.txt
← back to cnn.com · change detected 23d ago (2026-10-15T00:17:21Z) · modified +2 −1
What changed
- Blocked user-agent ‘ExaSearchBot’ entirely
- Blocked user-agent ‘ExaUser’ entirely
- 🤖 AIUnblocked Google-Extended (Google) - rules removed
Diff old (2026-09-25T17:22:55Z) → new (2026-10-15T00:17:21Z)
| @@ -38,10 +38,11 @@ | ||
| 38 | 38 | User-agent: DuckAssistBot |
| 39 | 39 | User-agent: EchoboxBot |
| 40 | 40 | User-agent: Exabot |
| 41 | + User-agent: ExaSearchBot | |
| 42 | + User-agent: ExaUser | |
| 41 | 43 | User-agent: FacebookBot |
| 42 | 44 | User-agent: FriendlyCrawler |
| 43 | 45 | User-agent: GPTBot |
| 44 | - User-agent: Google-Extended | |
| 45 | 46 | User-agent: GoogleOther |
| 46 | 47 | User-agent: GoogleOther-Image |
| 47 | 48 | User-agent: GoogleOther-Video |