El País / robots.txt
← back to elpais.com · change detected 1d ago (2026-06-24T22:21:31Z) · modified +14 −0
What changed
- 🤖 AIBlocked Claude-SearchBot (Anthropic) entirely
- Blocked user-agent ‘Comet’ entirely
- 🤖 AIBlocked Perplexity-User (Perplexity) entirely
- 🤖 AIBlocked PerplexityBot (Perplexity) entirely
Diff old (2026-06-20T01:10:30Z) → new (2026-06-24T22:21:31Z)
| @@ -36,6 +36,12 @@ | ||
| 36 | 36 | Disallow: / |
| 37 | 37 | |
| 38 | 38 | User-agent: Claude-User |
| 39 | + Disallow: / | |
| 40 | + | |
| 41 | + User-agent: Claude-SearchBot | |
| 42 | + Disallow: / | |
| 43 | + | |
| 44 | + User-agent: Comet | |
| 39 | 45 | Disallow: / |
| 40 | 46 | |
| 41 | 47 | User-agent: CloudFlare-AlwaysOnline |
| @@ -191,6 +197,12 @@ | ||
| 191 | 197 | User-agent: ProSitemaps |
| 192 | 198 | Disallow: / |
| 193 | 199 | |
| 200 | + User-agent: PerplexityBot | |
| 201 | + Disallow: / | |
| 202 | + | |
| 203 | + User-agent: Perplexity-User | |
| 204 | + Disallow: / | |
| 205 | + | |
| 194 | 206 | User-agent: proximic |
| 195 | 207 | Disallow: / |
| 196 | 208 | |
| @@ -283,6 +295,8 @@ | ||
| 283 | 295 | |
| 284 | 296 | User-agent: YandexUserproxy |
| 285 | 297 | Disallow: / |
| 298 | + | |
| 299 | + # OAI-AdsBot se mantiene permitido y no se bloquea. | |
| 286 | 300 | |
| 287 | 301 | User-agent: * |
| 288 | 302 | Disallow: /pruebas/ |