BBC / robots.txt
← back to bbc.com · change detected 6d ago (2026-07-29T16:38:54Z) · modified +7 −1
What changed
- 🤖 AIAdded Allow: /storyworks for ChatGPT-User
- 🤖 AIAdded Allow: /storyworks/ for ChatGPT-User
- 🤖 AIAdded Allow: /storyworks for Google-Extended
- 🤖 AIAdded Allow: /storyworks/ for Google-Extended
- 🤖 AIAdded Allow: /storyworks for GPTBot
- 🤖 AIAdded Allow: /storyworks/ for GPTBot
Diff old (2026-07-17T12:07:56Z) → new (2026-07-29T16:38:54Z)
| @@ -1,5 +1,5 @@ | ||
| 1 | 1 | |
| 2 | - # version: df99009430a1a8f8979042c24e21e9fecd8b21cd | |
| 2 | + # version: a294511aea2df10f410588a60a9157e46ab994a2 | |
| 3 | 3 | # The BBC's Terms of Use: https://www.bbc.co.uk/terms |
| 4 | 4 | # - Explain the rules for using our services |
| 5 | 5 | # - Tell you what you can do with our content |
| @@ -148,12 +148,18 @@ | ||
| 148 | 148 | |
| 149 | 149 | User-agent: GPTBot |
| 150 | 150 | Disallow: / |
| 151 | + Allow: /storyworks | |
| 152 | + Allow: /storyworks/ | |
| 151 | 153 | |
| 152 | 154 | User-agent: ChatGPT-User |
| 153 | 155 | Disallow: / |
| 156 | + Allow: /storyworks | |
| 157 | + Allow: /storyworks/ | |
| 154 | 158 | |
| 155 | 159 | User-agent: Google-Extended |
| 156 | 160 | Disallow: / |
| 161 | + Allow: /storyworks | |
| 162 | + Allow: /storyworks/ | |
| 157 | 163 | |
| 158 | 164 | User-Agent: PerplexityBot |
| 159 | 165 | Disallow: / |