NeuralCrawl

Progressive / robots.txt

← back to progressive.com · change detected 9d ago (2026-10-29T11:22:58Z) · modified +1 −0

What changed

Diff old (2026-10-22T15:00:52Z) → new (2026-10-29T11:22:58Z)

@@ -28,6 +28,7 @@
2828  Disallow:/agent/local-agent/
2929  Disallow:/*?slssid=
3030  Disallow:/*?subid=
31+ Disallow:/Log/LogVisitorData
3132 
3233  User-agent:AdsBot-Google
3334