Westpac / robots.txt snapshot
← back to westpac.com.au · fetched 2026-08-20T12:04:45Z (2mo ago) · HTTP 200 · 964 bytes · sha256 7eaaee7b7b68d992 · raw
final URL: https://westpac.com.au/robots.txt
| 1 | # Allow all crawlers |
| 2 | User-agent: * |
| 3 | Allow: / |
| 4 | |
| 5 | # Keep these tool endpoints crawlable all crawlers |
| 6 | Allow: /content/public/wbc/en/_services/serialiser/calculator/ |
| 7 | Allow: /content/public/wbc/en/_services/calcs-forms-tools/ |
| 8 | |
| 9 | # Restricted areas for all crawlers |
| 10 | Disallow: /search/ |
| 11 | Disallow: /branch-locator/ |
| 12 | Disallow: /_services/ |
| 13 | Disallow: /_services/serialiser/ |
| 14 | Disallow: /content/public/wbc/en/_services/serialiser/ |
| 15 | Disallow: /content/dam/public/wbc/introducernet/ |
| 16 | Disallow: /content/public/wbc/en/deeplink-web-redirect.html |
| 17 | Disallow: /locateus-ios* |
| 18 | Disallow: /locateus-android* |
| 19 | Disallow: /disclaimer/ |
| 20 | Disallow: /footer-cta/ |
| 21 | Disallow: /test-aem-1817/ |
| 22 | Disallow: /test/ |
| 23 | Disallow: /*.doc |
| 24 | Disallow: /*.docx |
| 25 | |
| 26 | # Global sitemap declarations |
| 27 | Sitemap: https://www.westpac.com.au/sitemap.xml |
| 28 | Sitemap: https://www.westpac.com.au/sitemap-65.xml |
| 29 | Sitemap: https://www.westpac.com.au/news/sitemap.xml |
| 30 | Sitemap: https://www.westpac.com.au/content/dam/public/wbc/seo/sitemap-locator.xml |