NeuralCrawl

Molson Coors / robots.txt snapshot

← back to molsoncoors.com · fetched 2026-06-20T01:10:31Z (18h ago) · HTTP 200 · 3297 bytes · sha256 89d1337b90b2151d · raw

final URL: https://www.molsoncoors.com/robots.txt

1#
2# robots.txt
3#
4# This file is to prevent the crawling and indexing of certain parts
5# of your site by web crawlers and spiders run by sites like Yahoo!
6# and Google. By telling these "robots" where not to go on your site,
7# you save bandwidth and server resources.
8#
9# This file will be ignored unless it is at the root of your host:
10# Used: http://example.com/robots.txt
11# Ignored: http://example.com/site/robots.txt
12#
13# For more information about the robots.txt standard, see:
14# http://www.robotstxt.org/robotstxt.html
15
16User-agent: *
17# CSS, JS, Images
18Allow: /core/*.css$
19Allow: /core/*.css?
20Allow: /core/*.js$
21Allow: /core/*.js?
22Allow: /core/*.gif
23Allow: /core/*.jpg
24Allow: /core/*.jpeg
25Allow: /core/*.png
26Allow: /core/*.svg
27Allow: /profiles/*.css$
28Allow: /profiles/*.css?
29Allow: /profiles/*.js$
30Allow: /profiles/*.js?
31Allow: /profiles/*.gif
32Allow: /profiles/*.jpg
33Allow: /profiles/*.jpeg
34Allow: /profiles/*.png
35Allow: /profiles/*.svg
36Allow: /modules/*.css$
37Allow: /modules/*.css?
38Allow: /modules/*.js$
39Allow: /modules/*.js?
40Allow: /modules/*.gif
41Allow: /modules/*.jpg
42Allow: /modules/*.jpeg
43Allow: /modules/*.png
44Allow: /modules/*.svg
45Allow: /themes/*.css$
46Allow: /themes/*.css?
47Allow: /themes/*.js$
48Allow: /themes/*.js?
49Allow: /themes/*.gif
50Allow: /themes/*.jpg
51Allow: /themes/*.jpeg
52Allow: /themes/*.png
53Allow: /themes/*.svg
54# Directories
55Disallow: /core/
56Disallow: /profiles/
57Disallow: /modules/
58Disallow: /themes/
59# Files
60Disallow: /README.md
61Disallow: /composer/Metapackage/README.txt
62Disallow: /composer/Plugin/ProjectMessage/README.md
63Disallow: /composer/Plugin/Scaffold/README.md
64Disallow: /composer/Plugin/VendorHardening/README.txt
65Disallow: /composer/Template/README.txt
66Disallow: /modules/README.txt
67Disallow: /sites/README.txt
68Disallow: /themes/README.txt
69Disallow: /web.config
70# Paths (clean URLs)
71Disallow: /admin/
72Disallow: /*/admin/
73Disallow: /comment/reply/
74Disallow: /*/comment/reply/
75Disallow: /filter/tips
76Disallow: /*/filter/tips
77Disallow: /node/add/
78Disallow: /*/node/add/
79Disallow: /search/
80Disallow: /*/search/
81Disallow: /user/register
82Disallow: /*/user/register
83Disallow: /user/password
84Disallow: /*/user/password
85Disallow: /user/login
86Disallow: /*/user/login
87Disallow: /user/logout
88Disallow: /*/user/logout
89Disallow: /media/oembed
90Disallow: /*/media/oembed
91# SSO/OAuth helper endpoints (crawl hygiene)
92Disallow: /sso-login
93Disallow: /*/sso-login
94Disallow: /moLogin
95Disallow: /*/moLogin
96Disallow: /mo_callback/
97Disallow: /*/mo_callback/
98Disallow: /samllogin/
99Disallow: /*/samllogin/
100Disallow: /samlassertion
101Disallow: /*/samlassertion
102Disallow: /saml_login
103Disallow: /*/saml_login
104Disallow: /mo/oauth2/userinfo
105Disallow: /*/mo/oauth2/userinfo
106# Paths (no clean URLs)
107Disallow: /index.php/admin/
108Disallow: /index.php/comment/reply/
109Disallow: /index.php/filter/tips
110Disallow: /index.php/node/add/
111Disallow: /index.php/search/
112Disallow: /index.php/user/password
113Disallow: /index.php/user/register
114Disallow: /index.php/user/login
115Disallow: /index.php/user/logout
116Disallow: /index.php/media/oembed
117Disallow: /index.php/*/media/oembed
118Disallow: /index.php/sso-login
119Disallow: /index.php/moLogin
120Disallow: /index.php/mo_callback/
121Disallow: /index.php/samllogin/
122Disallow: /index.php/samlassertion
123Disallow: /index.php/saml_login
124Disallow: /index.php/mo/oauth2/userinfo