NRG Energy / robots.txt
← back to nrg.com · change detected 1mo ago (2026-10-04T02:35:37Z) · modified +5 −11
What changed
- Added Disallow: */search for *
- Added Disallow: *?keyword=* for *
- Removed Disallow: */campaigns for *
- Removed Disallow: /*.html/plans/ for *
- Removed Disallow: /business/*.html/plans/ for *
- Removed Disallow: /contact-us/thank-you.html for *
- Removed Disallow: /content/nrg/lp for *
- Removed Disallow: /search for *
- Added sitemap https://www.nrg.com/en.sitemap.xml
- Removed sitemap https://www.nrg.com/sitemap.xml
Diff old (2026-06-20T01:10:30Z) → new (2026-10-04T02:35:37Z)
| @@ -1,13 +1,7 @@ | ||
| 1 | - Sitemap: https://www.nrg.com/sitemap.xml | |
| 1 | + User-agent: * | |
| 2 | 2 | |
| 3 | - User-agent: * | |
| 4 | - Disallow: /search | |
| 5 | - Disallow: /contact-us/thank-you.html | |
| 6 | - Disallow: /*.html/plans/ | |
| 7 | - Disallow: /business/*.html/plans/ | |
| 8 | - Disallow: */campaigns | |
| 9 | - Disallow: /content/nrg/lp | |
| 10 | - | |
| 3 | + Disallow: */search | |
| 4 | + Disallow: *?keyword=* | |
| 11 | 5 | |
| 12 | 6 | User-agent: Baiduspider |
| 13 | 7 | User-agent: Baiduspider-video |
| @@ -16,7 +10,7 @@ | ||
| 16 | 10 | User-agent: Baiduspider-cpro |
| 17 | 11 | User-agent: Baiduspider-favo |
| 18 | 12 | User-agent: Baiduspider-news |
| 13 | + User-agent: PetalBot | |
| 19 | 14 | Disallow: / |
| 20 | 15 | |
| 21 | - User-agent: PetalBot | |
| 22 | - Disallow: / | |
| 16 | + Sitemap: https://www.nrg.com/en.sitemap.xml |