Reuters / robots.txt
← back to reuters.com · change detected 3d ago (2026-09-15T14:34:35Z) · modified +0 −1
What changed
- Removed Disallow: /jp/ for AASA-Bot
- Removed Disallow: /jp/ for ADmantX
- Removed Disallow: /jp/ for AdmiralBot
- Removed Disallow: /jp/ for AdsBot-Google
- Removed Disallow: /jp/ for AdsBot-Google-Mobile
- Removed Disallow: /jp/ for AmazonAdBot
- Removed Disallow: /jp/ for Amzn-User
- Removed Disallow: /jp/ for Applebot
- Removed Disallow: /jp/ for AppleNewsBot
- Removed Disallow: /jp/ for Bingbot
- Removed Disallow: /jp/ for BingPreview
- Removed Disallow: /jp/ for BrightEdge
- Removed Disallow: /jp/ for BrightEdgeOnCrawl
- Removed Disallow: /jp/ for CensysInspect
- 🤖 AIRemoved Disallow: /jp/ for ChatGPT-User
- Removed Disallow: /jp/ for Cision
- Removed Disallow: /jp/ for Clickagy
- Removed Disallow: /jp/ for Concert
- Removed Disallow: /jp/ for ContextualBot
- Removed Disallow: /jp/ for CriteoBot
- Removed Disallow: /jp/ for DatadogSynthetics
- Removed Disallow: /jp/ for datadome-pageprotect-scanner
- Removed Disallow: /jp/ for Discordbot
- Removed Disallow: /jp/ for doubleverify
- Removed Disallow: /jp/ for ElevenlabsBot
- Removed Disallow: /jp/ for Embedly
- Removed Disallow: /jp/ for facebookexternalhit
- Removed Disallow: /jp/ for Google-Display-Ads-Bot
- Removed Disallow: /jp/ for Google-InspectionTool
- Removed Disallow: /jp/ for Google-Read-Aloud
- Removed Disallow: /jp/ for Google-Safety
- Removed Disallow: /jp/ for Google-Site-Verification
- Removed Disallow: /jp/ for Googlebot
- Removed Disallow: /jp/ for Googlebot Smartphone
- Removed Disallow: /jp/ for Googlebot-News
- Removed Disallow: /jp/ for GoogleOther
- Removed Disallow: /jp/ for GTmetrix
- Removed Disallow: /jp/ for GumGumBot
- Removed Disallow: /jp/ for ias_crawler
- Removed Disallow: /jp/ for Iframely
- Removed Disallow: /jp/ for leiki
- Removed Disallow: /jp/ for LinkedInBot
- Removed Disallow: /jp/ for LinkTiger
- Removed Disallow: /jp/ for Mantisbot
- Removed Disallow: /jp/ for Mediapartners-Google
- Removed Disallow: /jp/ for meta-externalads
- 🤖 AIRemoved Disallow: /jp/ for meta-webindexer
- Removed Disallow: /jp/ for MicrosoftPreview
- Removed Disallow: /jp/ for MJ12bot
- Removed Disallow: /jp/ for Moreover
- ... and 36 more change(s)
Diff old (2026-09-11T00:58:05Z) → new (2026-09-15T14:34:35Z)
| @@ -118,7 +118,6 @@ | ||
| 118 | 118 | Disallow: /pt/ |
| 119 | 119 | Disallow: /de/ |
| 120 | 120 | Disallow: /latam/ |
| 121 | - Disallow: /jp/ | |
| 122 | 121 | Disallow: /account/subscribe/payment |
| 123 | 122 | |
| 124 | 123 | User-agent: Applebot-Extended |