BP / robots.txt
← back to bp.com · change detected 1mo ago (2026-09-09T12:27:15Z) · modified +0 −2
What changed
- Removed Disallow: /en_us/united-states/home/referencing.html for *
- Removed Disallow: /en_us/united-states/home/referencing.html for Algolia Crawler
Diff old (2026-06-20T01:10:30Z) → new (2026-09-09T12:27:15Z)
| @@ -4,7 +4,6 @@ | ||
| 4 | 4 | Disallow: /*.html?* |
| 5 | 5 | Disallow: /en_gb/united-kingdom/media/press-releases/bpme-launch.html |
| 6 | 6 | Disallow: /en_gb/united-kingdom/home/references.html |
| 7 | - Disallow: /en_us/united-states/home/referencing.html | |
| 8 | 7 | Disallow: /en_gb/united-kingdom/home/products-and-services/bp-fleet/privacy-statement.html |
| 9 | 8 | Disallow: /en_gb/united-kingdom/home/products-and-services/bp-fleet/legal-notice.html |
| 10 | 9 | Disallow: /en_gb/united-kingdom/home/products-and-services/bp-fleet/contactus.html |
| @@ -55,7 +54,6 @@ | ||
| 55 | 54 | User-agent: Algolia Crawler |
| 56 | 55 | Disallow: |
| 57 | 56 | Disallow: /en_gb/united-kingdom/media/press-releases/bpme-launch.html |
| 58 | - Disallow: /en_us/united-states/home/referencing.html | |
| 59 | 57 | Disallow: /narrative2018 |
| 60 | 58 | Disallow: /en_gb/united-kingdom/home/references.html |
| 61 | 59 |