NeuralCrawl

Marsh McLennan / robots.txt snapshot

← back to marshmclennan.com · fetched 2026-08-26T14:01:29Z (2mo ago) · HTTP 200 · 633 bytes · sha256 0057e4888c8ccd49 · raw

final URL: https://www.marsh.com/robots.txt

1User-agent: *
2
3Disallow: /*.html/content/
4Disallow: /sharedcontent/
5Disallow: /*/_jcr_content/
6Disallow: /*/jcr:content/
7Disallow: /*/designs/
8Disallow: /*/etc/tags/
9Disallow: /*/registration-form/request-submitted.html
10Disallow: /*/registration-form/email-not-validated.html
11Disallow: /*/registration-form/email-sent.html
12
13Disallow: /libs/cq/security/userinfo.json*
14Disallow: *.cookiepolicy.json*
15Disallow: /etc/segmentation.segment.js?_=*
16
17Disallow: /*/search.html
18Disallow: /*/forms.html
19Disallow: /*/campaigns.html
20Disallow: /*/register.html
21Disallow: /*/registration-form.html
22Disallow: /*/confirmation.html
23Disallow: /*/moi?=*