Financial Times / robots.txt
← back to ft.com · change detected 7d ago (2026-10-30T17:56:11Z) · modified +3 −0
What changed
- 🤖 AIBlocked Meta-WebIndexer (Meta) entirely
Diff old (2026-07-23T14:11:06Z) → new (2026-10-30T17:56:11Z)
| @@ -86,3 +86,6 @@ | ||
| 86 | 86 | |
| 87 | 87 | User-agent: Meta-ExternalFetcher |
| 88 | 88 | Disallow: / |
| 89 | + | |
| 90 | + User-agent: Meta-WebIndexer | |
| 91 | + Disallow: / |