Der Spiegel / robots.txt
← back to spiegel.de · change detected 24d ago (2026-10-13T11:46:10Z) · modified +0 −2
What changed
- 🤖 AIUnblocked Amazonbot (Amazon) - rules removed
Diff old (2026-09-29T12:52:06Z) → new (2026-10-13T11:46:10Z)
| @@ -99,8 +99,6 @@ | ||
| 99 | 99 | User-agent: Applebot-Extended |
| 100 | 100 | Disallow: / |
| 101 | 101 | User-agent: MistralAI-User |
| 102 | - Disallow: / | |
| 103 | - User-agent: Amazonbot | |
| 104 | 102 | Disallow: / |
| 105 | 103 | User-agent: AI2Bot |
| 106 | 104 | Disallow: / |