The New York Times / robots.txt
← back to nytimes.com · change detected 2mo ago (2026-08-12T21:29:54Z) · modified +1 −1
What changed
- Formatting/comment-only change (no semantic difference)
Diff old (2026-06-20T01:10:30Z) → new (2026-08-12T21:29:54Z)
| @@ -140,7 +140,7 @@ | ||
| 140 | 140 | User-agent: AliyunSecBot |
| 141 | 141 | Disallow: / |
| 142 | 142 | |
| 143 | - User-agent: AmazonBot | |
| 143 | + User-agent: Amazonbot | |
| 144 | 144 | Disallow: /wirecutter/ |
| 145 | 145 | |
| 146 | 146 | User-agent: anthropic-ai |