NeuralCrawl

The Associated Press / robots.txt

← back to apnews.com · change detected 15d ago (2026-10-22T21:05:52Z) · modified +2 −0

What changed

Diff old (2026-10-22T02:51:55Z) → new (2026-10-22T21:05:52Z)

@@ -18,6 +18,8 @@
1818  Disallow: /press-release/*
1919  Disallow: /*/widgets/
2020  Disallow: /*/widget/
21+
22+ # Monitoring Test
2123 
2224  Sitemap: https://apnews.com/ap-sitemap.xml
2325  Sitemap: https://apnews.com/news-sitemap-content.xml