Al Jazeera / robots.txt
← back to aljazeera.com · change detected 1mo ago (2026-06-29T17:50:44Z) · modified +1 −1
What changed
- Added Disallow: /*/liveblog/*?*update=* for *
- Removed Disallow: /*/liveblog/2026/6/*?*update=* for *
Diff old (2026-06-20T01:10:30Z) → new (2026-06-29T17:50:44Z)
| @@ -17,7 +17,7 @@ | ||
| 17 | 17 | Allow: /search/$ |
| 18 | 18 | Disallow: /search/ |
| 19 | 19 | Disallow: /home/search?q= |
| 20 | - Disallow: /*/liveblog/2026/6/*?*update=* | |
| 20 | + Disallow: /*/liveblog/*?*update=* | |
| 21 | 21 | |
| 22 | 22 | # Allow Rules |
| 23 | 23 |