The Associated Press / robots.txt
← back to apnews.com · change detected 23d ago (2026-10-15T18:35:45Z) · modified +2 −1
What changed
- Added Disallow: /*/scripts/ for *
- Added Disallow: /*/widget/ for *
- Removed Disallow: (empty) for *
Diff old (2026-10-08T22:07:02Z) → new (2026-10-15T18:35:45Z)
| @@ -1,5 +1,4 @@ | ||
| 1 | 1 | User-Agent: * |
| 2 | - Disallow: | |
| 3 | 2 | Disallow: *_ptid=* |
| 4 | 3 | Disallow: *?prx_t=* |
| 5 | 4 | Disallow: /zephr* |
| @@ -18,6 +17,8 @@ | ||
| 18 | 17 | Disallow: /api/v2/feed/ |
| 19 | 18 | Disallow: /press-release/* |
| 20 | 19 | Disallow: /*/widgets/ |
| 20 | + Disallow: /*/widget/ | |
| 21 | + Disallow: /*/scripts/ | |
| 21 | 22 | |
| 22 | 23 | Sitemap: https://apnews.com/ap-sitemap.xml |
| 23 | 24 | Sitemap: https://apnews.com/news-sitemap-content.xml |