BuzzFeed / robots.txt
← back to buzzfeed.com · change detected 5h ago (2026-06-25T22:38:12Z) · modified +1 −0
What changed
- 🤖 AIAdded Allow: /rss-feeds/feeds/bf-amazon-ai-grounding.extended.xml for Amazonbot
Diff old (2026-06-20T01:10:30Z) → new (2026-06-25T22:38:12Z)
| @@ -46,6 +46,7 @@ | ||
| 46 | 46 | Disallow: /search/ |
| 47 | 47 | |
| 48 | 48 | User-agent: Amazonbot |
| 49 | + Allow: /rss-feeds/feeds/bf-amazon-ai-grounding.extended.xml | |
| 49 | 50 | Disallow: / |
| 50 | 51 | |
| 51 | 52 | User-agent: anthropic-ai |