NeuralCrawl

The Associated Press / robots.txt

← back to apnews.com · change detected 16d ago (2026-10-22T02:51:55Z) · modified +0 −1

What changed

Diff old (2026-10-15T18:35:45Z) → new (2026-10-22T02:51:55Z)

@@ -18,7 +18,6 @@
1818  Disallow: /press-release/*
1919  Disallow: /*/widgets/
2020  Disallow: /*/widget/
21- Disallow: /*/scripts/
2221 
2322  Sitemap: https://apnews.com/ap-sitemap.xml
2423  Sitemap: https://apnews.com/news-sitemap-content.xml