Corriere della Sera / robots.txt
← back to corriere.it · change detected 1mo ago (2026-08-07T13:42:41Z) · modified +6 −0
What changed
- Blocked user-agent ‘Baiduspider’ entirely
- Blocked user-agent ‘NovriumRanker’ entirely
- Blocked user-agent ‘Stucco5000’ entirely
Diff old (2026-06-29T05:43:55Z) → new (2026-08-07T13:42:41Z)
| @@ -34,6 +34,12 @@ | ||
| 34 | 34 | User-agent: YouBot |
| 35 | 35 | Disallow: / |
| 36 | 36 | User-agent: Timpibot |
| 37 | + Disallow: / | |
| 38 | + User-agent: NovriumRanker | |
| 39 | + Disallow: / | |
| 40 | + User-agent: Stucco5000 | |
| 41 | + Disallow: / | |
| 42 | + User-agent: Baiduspider | |
| 37 | 43 | Disallow: / |
| 38 | 44 | # Fine Bot Exclusion Protocol |
| 39 | 45 |