NeuralCrawl

MetLife / robots.txt snapshot

← back to metlife.com · fetched 2026-07-28T10:03:18Z (9d ago) · HTTP 200 · 895 bytes · sha256 ea3565a3d0854633 · raw

final URL: https://www.metlife.com/robots.txt

1User-agent: *
2Disallow: /investments$
3Disallow: /investments/
4Disallow: /sitemap/_jcr_content/root/sitemap.xml
5Disallow: /*aemform.iframe.html*
6Disallow: /uat/
7Disallow: /smoketest/
8Disallow: /pagenotfound/
9Disallow: /*.pdf
10Disallow: /*.pdf$
11Disallow: https://origin-intl.metlife.com/*
12Disallow: https://target.www.metlife.com/*
13Disallow: https://origin-target.metlife.com/*
14User-agent: prodGSA
15Disallow: /system/
16Disallow: /MCOMDefault/
17Disallow: /WPSAssets/
18Disallow: /navimages/
19Disallow: /*.css$
20Disallow: /*.js$
21Disallow: /pagenotfound/
22User-agent: drGSA
23Disallow: /system/
24Disallow: /MCOMDefault/
25Disallow: /WPSAssets/
26Disallow: /navimages/
27Disallow: /*.css$
28Disallow: /*.js$
29Disallow: /pagenotfound/
30User-agent: devqaGSA
31Disallow: /system/
32Disallow: /MCOMDefault/
33Disallow: /WPSAssets/
34Disallow: /navimages/
35Disallow: /*.css$
36Disallow: /*.js$
37Sitemap: https://www.metlife.com/sitemap.xml