Al Jazeera / robots.txt
← back to aljazeera.com · change detected 8d ago (2026-10-29T23:30:26Z) · modified +2 −1
What changed
- Added Disallow: /*&update= for *
- Added Disallow: /*?update= for *
- Removed Disallow: /*/liveblog/*?*update=* for *
Diff old (2026-07-13T16:47:36Z) → new (2026-10-29T23:30:26Z)
| @@ -17,7 +17,8 @@ | ||
| 17 | 17 | Allow: /search/$ |
| 18 | 18 | Disallow: /search/ |
| 19 | 19 | Disallow: /home/search?q= |
| 20 | - Disallow: /*/liveblog/*?*update=* | |
| 20 | + Disallow: /*?update= | |
| 21 | + Disallow: /*&update= | |
| 21 | 22 | Disallow: /*?gb= |
| 22 | 23 | Disallow: /*&gb= |
| 23 | 24 | Disallow: /*?traffic_source= |