Westlake / robots.txt
← back to westlake.com · change detected 2mo ago (2026-08-25T01:27:49Z) · modified +3 −1
What changed
- Removed Disallow: /web.config for *
- Added sitemap https://www.westlake.com/sitemap.xml
Diff old (2026-06-20T01:10:30Z) → new (2026-08-25T01:27:49Z)
| @@ -12,6 +12,9 @@ | ||
| 12 | 12 | # |
| 13 | 13 | # For more information about the robots.txt standard, see: |
| 14 | 14 | # http://www.robotstxt.org/robotstxt.html |
| 15 | + | |
| 16 | + # Sitemap location must be an absolute URL. | |
| 17 | + Sitemap: https://www.westlake.com/sitemap.xml | |
| 15 | 18 | |
| 16 | 19 | User-agent: * |
| 17 | 20 | # CSS, JS, Images |
| @@ -46,7 +49,6 @@ | ||
| 46 | 49 | Disallow: /modules/README.txt |
| 47 | 50 | Disallow: /sites/README.txt |
| 48 | 51 | Disallow: /themes/README.txt |
| 49 | - Disallow: /web.config | |
| 50 | 52 | # Paths (clean URLs) |
| 51 | 53 | Disallow: /admin/ |
| 52 | 54 | Disallow: /comment/reply/ |