NeuralCrawl

CME Group / robots.txt

← back to cmegroup.com · change detected 15d ago (2026-10-23T03:13:22Z) · modified +2 −1

What changed

Diff old (2026-08-15T16:34:17Z) → new (2026-10-23T03:13:22Z)

@@ -1,4 +1,6 @@
11  User-Agent: *
2+ Sitemap: https://www.cmegroup.com/sitemap.xml
3+ Sitemap: https://www.cmegroup.com/eds/sitemap.xml
24  Sitemap: https://www.cmegroup.com/sitemap_index.xml
35  Disallow: /cgi-bin/
46  Disallow: /javascript/
@@ -360,7 +362,6 @@
360362  Disallow: /tools-information/lookups/advisories/market-data/banner/
361363  Disallow: /tools-information/rightnav/
362364  Disallow: /tools-information/webhelp/account-manager/
363- Disallow: /tools-information/webhelp/cme-direct/
364365  Disallow: /tools-information/webhelp/no-index/
365366  Disallow: /tools-information/webhelp/rightnav/
366367  Disallow: /tools-information/webhelp/cme-auction-crude-oil-buyer/Content/Home.html