CNN / robots.txt
← back to cnn.com · change detected 1mo ago (2026-09-24T23:05:48Z) · modified +4 −0
What changed
- Blocked user-agent ‘Shap-User’ entirely
- Blocked user-agent ‘ShapBot’ entirely
Diff old (2026-06-20T01:10:30Z) → new (2026-09-24T23:05:48Z)
| @@ -127,3 +127,7 @@ | ||
| 127 | 127 | Disallow: /webview/ |
| 128 | 128 | User-agent: Googlebot-News |
| 129 | 129 | Disallow: /sponsor |
| 130 | + User-agent: ShapBot | |
| 131 | + Disallow: / | |
| 132 | + User-agent: Shap-User | |
| 133 | + Disallow: / |