NPR / robots.txt
← back to npr.org · change detected 1mo ago (2026-09-22T16:26:18Z) · modified +6 −0
What changed
- 🤖 AIBlocked Claude-SearchBot (Anthropic) entirely
- 🤖 AIBlocked Claude-User (Anthropic) entirely
Diff old (2026-06-20T01:10:30Z) → new (2026-09-22T16:26:18Z)
| @@ -68,6 +68,12 @@ | ||
| 68 | 68 | Disallow: / |
| 69 | 69 | |
| 70 | 70 | User-agent: ClaudeBot |
| 71 | + Disallow: / | |
| 72 | + | |
| 73 | + User-agent: Claude-SearchBot | |
| 74 | + Disallow: / | |
| 75 | + | |
| 76 | + User-agent: Claude-User | |
| 71 | 77 | Disallow: / |
| 72 | 78 | |
| 73 | 79 | User-agent: Claude-Web |