The Independent / robots.txt
← back to independent.co.uk · change detected 24d ago (2026-07-13T22:52:30Z) · modified +2 −1
What changed
- Formatting/comment-only change (no semantic difference)
Diff old (2026-07-13T16:47:36Z) → new (2026-07-13T22:52:30Z)
| @@ -1,3 +1,5 @@ | ||
| 1 | + | |
| 2 | + | |
| 1 | 3 | |
| 2 | 4 | User-agent: * |
| 3 | 5 | |
| @@ -54,7 +56,6 @@ | ||
| 54 | 56 | |
| 55 | 57 | User-agent: Nutch |
| 56 | 58 | Disallow: / |
| 57 | - | |
| 58 | 59 | Sitemap: https://www.independent.co.uk/sitemaps/googlenews |
| 59 | 60 | Sitemap: https://www.independent.co.uk/sitemap.xml |
| 60 | 61 |