Reuters / robots.txt
← back to reuters.com · change detected 1mo ago (2026-09-23T22:52:11Z) · modified +1 −0
What changed
- Added Disallow: /*/site-search/ for AASA-Bot
- Added Disallow: /*/site-search/ for ADmantX
- Added Disallow: /*/site-search/ for AdmiralBot
- Added Disallow: /*/site-search/ for AdsBot-Google
- Added Disallow: /*/site-search/ for AdsBot-Google-Mobile
- Added Disallow: /*/site-search/ for AmazonAdBot
- Added Disallow: /*/site-search/ for Amzn-User
- Added Disallow: /*/site-search/ for Applebot
- Added Disallow: /*/site-search/ for AppleNewsBot
- Added Disallow: /*/site-search/ for Bingbot
- Added Disallow: /*/site-search/ for BingPreview
- Added Disallow: /*/site-search/ for BrightEdge
- Added Disallow: /*/site-search/ for BrightEdgeOnCrawl
- Added Disallow: /*/site-search/ for CensysInspect
- 🤖 AIAdded Disallow: /*/site-search/ for ChatGPT-User
- Added Disallow: /*/site-search/ for Cision
- Added Disallow: /*/site-search/ for Clickagy
- Added Disallow: /*/site-search/ for Concert
- Added Disallow: /*/site-search/ for ContextualBot
- Added Disallow: /*/site-search/ for CriteoBot
- Added Disallow: /*/site-search/ for DatadogSynthetics
- Added Disallow: /*/site-search/ for datadome-pageprotect-scanner
- Added Disallow: /*/site-search/ for Discordbot
- Added Disallow: /*/site-search/ for doubleverify
- Added Disallow: /*/site-search/ for ElevenlabsBot
- Added Disallow: /*/site-search/ for Embedly
- Added Disallow: /*/site-search/ for facebookexternalhit
- Added Disallow: /*/site-search/ for Google-Display-Ads-Bot
- Added Disallow: /*/site-search/ for Google-InspectionTool
- Added Disallow: /*/site-search/ for Google-Read-Aloud
- Added Disallow: /*/site-search/ for Google-Safety
- Added Disallow: /*/site-search/ for Google-Site-Verification
- Added Disallow: /*/site-search/ for Googlebot
- Added Disallow: /*/site-search/ for Googlebot Smartphone
- Added Disallow: /*/site-search/ for Googlebot-News
- Added Disallow: /*/site-search/ for GoogleOther
- Added Disallow: /*/site-search/ for GTmetrix
- Added Disallow: /*/site-search/ for GumGumBot
- Added Disallow: /*/site-search/ for ias_crawler
- Added Disallow: /*/site-search/ for Iframely
- Added Disallow: /*/site-search/ for leiki
- Added Disallow: /*/site-search/ for LinkedInBot
- Added Disallow: /*/site-search/ for LinkTiger
- Added Disallow: /*/site-search/ for Mantisbot
- Added Disallow: /*/site-search/ for Mediapartners-Google
- Added Disallow: /*/site-search/ for meta-externalads
- 🤖 AIAdded Disallow: /*/site-search/ for meta-webindexer
- Added Disallow: /*/site-search/ for MicrosoftPreview
- Added Disallow: /*/site-search/ for MJ12bot
- Added Disallow: /*/site-search/ for Moreover
- ... and 36 more change(s)
Diff old (2026-09-17T03:02:10Z) → new (2026-09-23T22:52:11Z)
| @@ -119,6 +119,7 @@ | ||
| 119 | 119 | Disallow: /de/ |
| 120 | 120 | Disallow: /latam/ |
| 121 | 121 | Disallow: /account/subscribe/payment |
| 122 | + Disallow: /*/site-search/ | |
| 122 | 123 | |
| 123 | 124 | User-agent: Applebot-Extended |
| 124 | 125 | Disallow: / |