De Telegraaf / robots.txt
← back to telegraaf.nl · change detected 2mo ago (2026-08-10T14:47:29Z) · modified +16 −1
What changed
- Blocked user-agent ‘Grok’ entirely
- Blocked user-agent ‘Grok-DeepSearch’ entirely
- Blocked user-agent ‘Grokbot’ entirely
- Blocked user-agent ‘xAI-Bot’ entirely
- Blocked user-agent ‘xAI-Grok’ entirely
Diff old (2026-06-24T16:12:44Z) → new (2026-08-10T14:47:29Z)
| @@ -1,4 +1,4 @@ | ||
| 1 | - # Generated: 2026-06-24T13:02:02.886Z | |
| 1 | + # Generated: 2026-08-10T08:26:49.822Z | |
| 2 | 2 | # Brand: telegraaf.nl | Environment: prod |
| 3 | 3 | |
| 4 | 4 | # All copyrights, neighbouring rights and database rights in the content and layout of this website/app are explicitly reserved and are for personal, non-commercial use only. |
| @@ -123,6 +123,21 @@ | ||
| 123 | 123 | User-agent: Youbot |
| 124 | 124 | Disallow: / |
| 125 | 125 | |
| 126 | + User-agent: Grok | |
| 127 | + Disallow: / | |
| 128 | + | |
| 129 | + User-agent: Grokbot | |
| 130 | + Disallow: / | |
| 131 | + | |
| 132 | + User-agent: Grok-DeepSearch | |
| 133 | + Disallow: / | |
| 134 | + | |
| 135 | + User-agent: xAI-Bot | |
| 136 | + Disallow: / | |
| 137 | + | |
| 138 | + User-agent: xAI-Grok | |
| 139 | + Disallow: / | |
| 140 | + | |
| 126 | 141 | # User-agent: Bingbot |
| 127 | 142 | # Disallow: / |
| 128 | 143 |