NBCUniversal / robots.txt
← back to nbcuniversal.com · change detected 1mo ago (2026-09-10T06:41:03Z) · modified +0 −9
What changed
- Removed all rules for user-agent ‘* Disallow:’
- Removed Allow: /newsroom for *
- Removed Allow: /newsroom/* for *
- Removed Disallow: /README.txt for *
- Removed sitemap https://www.nbcuniversal.com/sitemap.xml
Diff old (2026-06-20T01:10:31Z) → new (2026-09-10T06:41:03Z)
| @@ -37,7 +37,6 @@ | ||
| 37 | 37 | Disallow: /core/ |
| 38 | 38 | Disallow: /profiles/ |
| 39 | 39 | # Files |
| 40 | - Disallow: /README.txt | |
| 41 | 40 | Disallow: /README.md |
| 42 | 41 | Disallow: /composer/Metapackage/README.txt |
| 43 | 42 | Disallow: /composer/Plugin/ProjectMessage/README.md |
| @@ -72,11 +71,3 @@ | ||
| 72 | 71 | Disallow: /index.php/user/logout |
| 73 | 72 | Disallow: /index.php/media/oembed |
| 74 | 73 | Disallow: /index.php/*/media/oembed |
| 75 | - | |
| 76 | - # Newsroom | |
| 77 | - Allow: /newsroom | |
| 78 | - Allow: /newsroom/* | |
| 79 | - | |
| 80 | - # Sitemap | |
| 81 | - Sitemap: https://www.nbcuniversal.com/sitemap.xml | |
| 82 | - User-agent: * Disallow: |