Progressive / robots.txt
← back to progressive.com · change detected 30d ago (2026-10-08T16:07:06Z) · modified +4 −1
What changed
- Added Allow: /lp/motorcycle/ for *
- Added Disallow: /*.asp for *
- Added Disallow: /*lp/ for *
- Added Disallow: /agent/local-agent/ for *
- Removed Disallow: /lp/ for *
Diff old (2026-08-27T14:17:26Z) → new (2026-10-08T16:07:06Z)
| @@ -3,7 +3,8 @@ | ||
| 3 | 3 | Disallow:/af/ |
| 4 | 4 | Disallow:/api/ |
| 5 | 5 | Disallow:/dev/ |
| 6 | - Disallow:/lp/ | |
| 6 | + Disallow:/*lp/ | |
| 7 | + Allow:/lp/motorcycle/ | |
| 7 | 8 | Disallow:/benefits/ |
| 8 | 9 | Disallow:/error/ |
| 9 | 10 | Disallow:/popup/ |
| @@ -23,6 +24,8 @@ | ||
| 23 | 24 | Disallow:/aff-retrieve-quote/ |
| 24 | 25 | Disallow:/lp-agent/ |
| 25 | 26 | Disallow:/lifelanes/wp-content/ |
| 27 | + Disallow:/*.asp | |
| 28 | + Disallow:/agent/local-agent/ | |
| 26 | 29 | |
| 27 | 30 | User-agent:AdsBot-Google |
| 28 | 31 |