NeuralCrawl

AkzoNobel / robots.txt snapshot

← back to akzonobel.com · fetched 2026-06-20T01:10:30Z (15h ago) · HTTP 200 · 2851 bytes · sha256 dfb08ab8dc771aeb · raw

final URL: https://www.akzonobel.com/robots.txt

1#
2# robots.txt
3#
4# This file is to prevent the crawling and indexing of certain parts
5# of your site by web crawlers and spiders run by sites like Yahoo!
6# and Google. By telling these "robots" where not to go on your site,
7# you save bandwidth and server resources.
8#
9# This file will be ignored unless it is at the root of your host:
10# Used: http://example.com/robots.txt
11# Ignored: http://example.com/site/robots.txt
12#
13# For more information about the robots.txt standard, see:
14# http://www.robotstxt.org/robotstxt.html
15User-agent: *
16Crawl-delay: 10
17# CSS, JS, Images
18Allow: /misc/*.css$
19Allow: /misc/*.css?
20Allow: /misc/*.js$
21Allow: /misc/*.js?
22Allow: /misc/*.gif
23Allow: /misc/*.jpg
24Allow: /misc/*.jpeg
25Allow: /misc/*.png
26Allow: /modules/*.css$
27Allow: /modules/*.css?
28Allow: /modules/*.js$
29Allow: /modules/*.js?
30Allow: /modules/*.gif
31Allow: /modules/*.jpg
32Allow: /modules/*.jpeg
33Allow: /modules/*.png
34Allow: /profiles/*.css$
35Allow: /profiles/*.css?
36Allow: /profiles/*.js$
37Allow: /profiles/*.js?
38Allow: /profiles/*.gif
39Allow: /profiles/*.jpg
40Allow: /profiles/*.jpeg
41Allow: /profiles/*.png
42Allow: /themes/*.css$
43Allow: /themes/*.css?
44Allow: /themes/*.js$
45Allow: /themes/*.js?
46Allow: /themes/*.gif
47Allow: /themes/*.jpg
48Allow: /themes/*.jpeg
49Allow: /themes/*.png
50# Directories
51Disallow: /includes/
52Disallow: /misc/
53Disallow: /modules/
54Disallow: /profiles/
55Disallow: /scripts/
56Disallow: /themes/
57# Files
58Disallow: /CHANGELOG.txt
59Disallow: /cron.php
60Disallow: /INSTALL.mysql.txt
61Disallow: /INSTALL.pgsql.txt
62Disallow: /INSTALL.sqlite.txt
63Disallow: /install.php
64Disallow: /INSTALL.txt
65Disallow: /LICENSE.txt
66Disallow: /MAINTAINERS.txt
67Disallow: /update.php
68Disallow: /UPGRADE.txt
69Disallow: /xmlrpc.php
70# Paths (clean URLs)
71Disallow: /admin/
72Disallow: /comment/reply/
73Disallow: /filter/tips/
74Disallow: /node/add/
75Disallow: /search/
76Disallow: /user/register/
77Disallow: /user/password/
78Disallow: /user/login/
79Disallow: /user/logout/
80Disallow: /media-releases-and-features/*
81Disallow: /media-releases-and-features?*
82Disallow: /media-releases-and-features?page=*&*
83Disallow: /media-releases-and-features?page=0*
84Disallow: /bin/api/register
85Disallow: /bin/api/newsletter/subscribe
86Disallow: /bin/api/forgot-password
87Disallow: /bin/api/password
88Disallow: /bin/api/login
89Disallow: /bin/api/logout
90Disallow: /bin/api/contact-us
91Disallow: /bin/api/newsletter/subscribe/survey
92Disallow: /bin/api/register
93Disallow: /bin/api/compareProducts
94Disallow: /bin/api/search
95Disallow: /bin/api/login-shop
96Disallow: /bin/api/colorPopUp
97Disallow: /bin/api/*
98# Paths (no clean URLs)
99Disallow: /?q=admin/
100Disallow: /?q=comment/reply/
101Disallow: /?q=filter/tips/
102Disallow: /?q=node/add/
103Disallow: /?q=search/
104Disallow: /?q=user/password/
105Disallow: /?q=user/register/
106Disallow: /?q=user/login/
107Disallow: /?q=user/logout/
108# Sitemap
109Sitemap: https://www.akzonobel.com/en/sitemap.xml