The Economist / robots.txt
← back to economist.com · change detected 3mo ago (2026-08-06T13:18:41Z) · modified +2 −0
What changed
- 🤖 AIAdded Allow: /pro$ for Google-Extended
- 🤖 AIAdded Allow: /pro/ for Google-Extended
Diff old (2026-07-20T19:11:56Z) → new (2026-08-06T13:18:41Z)
| @@ -58,6 +58,8 @@ | ||
| 58 | 58 | # Google-Extended is used by the Gemini app for searches on |
| 59 | 59 | # behalf of the user but it also used for training |
| 60 | 60 | User-agent: Google-Extended |
| 61 | + Allow: /pro$ | |
| 62 | + Allow: /pro/ | |
| 61 | 63 | Disallow: / |
| 62 | 64 | |
| 63 | 65 | User-agent: anthropic-ai |