The Independent / robots.txt
← back to independent.co.uk · change detected 21d ago (2026-07-17T06:06:10Z) · modified +1 −2
What changed
- Formatting/comment-only change (no semantic difference)
Diff old (2026-07-13T22:52:30Z) → new (2026-07-17T06:06:10Z)
| @@ -1,5 +1,3 @@ | ||
| 1 | - | |
| 2 | - | |
| 3 | 1 | |
| 4 | 2 | User-agent: * |
| 5 | 3 | |
| @@ -56,6 +54,7 @@ | ||
| 56 | 54 | |
| 57 | 55 | User-agent: Nutch |
| 58 | 56 | Disallow: / |
| 57 | + | |
| 59 | 58 | Sitemap: https://www.independent.co.uk/sitemaps/googlenews |
| 60 | 59 | Sitemap: https://www.independent.co.uk/sitemap.xml |
| 61 | 60 |