Reuters / robots.txt
← back to reuters.com · change detected 1mo ago (2026-08-17T11:01:05Z) · modified +0 −1
What changed
- Removed all rules for user-agent ‘DuckDuckBot’
Diff old (2026-07-23T20:15:59Z) → new (2026-08-17T11:01:05Z)
| @@ -21,7 +21,6 @@ | ||
| 21 | 21 | User-agent: datadome-pageprotect-scanner |
| 22 | 22 | User-agent: Discordbot |
| 23 | 23 | User-agent: doubleverify |
| 24 | - User-agent: DuckDuckBot | |
| 25 | 24 | User-agent: ElevenlabsBot |
| 26 | 25 | User-agent: Embedly |
| 27 | 26 | User-agent: facebookexternalhit |