NeuralCrawl

AutoZone / robots.txt snapshot

← back to autozone.com · fetched 2026-06-20T01:10:31Z (18h ago) · HTTP 200 · 4260 bytes · sha256 fbb6a69980b67b05 · raw

final URL: https://www.autozone.com/robots.txt

1# RESEARCH TOOLS
2User-agent: BacklinkCrawler
3User-agent: GSiteCrawler
4User-agent: ip-web-crawler.com
5User-agent: ips-agent
6User-agent: Linkdex
7User-agent: linkdex.com
8User-agent: linkdex.com/v2.0
9User-agent: linkdex.com/v2.1
10User-agent: MJ12bot
11User-agent: SpyFu
12User-agent: Xenu
13User-agent: Xenu Link Sleuth
14User-agent: dotbot
15Disallow: /
16
17# Allow Microsoft crawlers
18User-agent: bingbot
19Allow: /
20
21User-agent: adidxbot
22Allow: /
23
24User-agent: MicrosoftPreview
25Allow: /
26
27# Inktomi / Yahoo
28User-agent: Slurp
29Disallow: /
30
31# INTERNATIONAL
32
33# United Kingdom
34User-agent: COMODOSpider/Nutch-1.2
35User-Agent: MJ12bot
36User-Agent: MJ12bot/v1.4.3
37User-agent: plukkie
38Disallow: /
39
40# Germany
41User-agent: Acoon
42User-agent: careerbot
43User-agent: findlinks/2.6
44User-agent: OpenWebIndex/Nutch-1.6
45User-agent: OpenWebIndex
46User-agent: SistrixCrawler
47User-agent: Sistrix
48User-agent: SISTRIX Crawler
49User-agent: SISTRIX
50Disallow: /
51
52# Sweden
53User-agent: psbot
54Disallow: /
55
56# Netherlands
57User-agent: KaloogaBot
58Disallow: /
59
60# Czech
61User-agent: SeznamBot
62User-agent: SeznamBot/1.0
63User-agent: SeznamBot/1.1
64User-agent: SeznamBot/3.0
65Disallow: /
66
67# China
68User-agent: Baiduspider
69User-agent: Baiduspider-video
70User-agent: Baiduspider-image
71User-agent: Baiduspider+
72User-agent: Baiduspider+(+http://www.baidu.com/search/spider.htm)
73User-agent: Baiduspider/2.0;+http://www.baidu.com/search/spider.html
74User-agent: Baiduspider/2.0
75User-agent: +Baiduspider
76User-agent: +Baiduspider/2.0
77User-agent: +Baiduspider/2.0;++http://www.baidu.com/search/spider.html
78User-agent: Mozilla/5.0(compatible; Baiduspider/2.0; +http://www.baidu.com/search/spider.html)
79User-agent: EasouSpider
80User-agent: JikeSpider
81User-agent: Sosospider+
82User-agent: Sosospider
83User-agent: Sosospider/2.0
84User-agent: 360Spider
85User-agent: YisouSpider
86User-agent: YoudaoBot/1.0
87User-agent: YoudaoBot/1.0
88User-agent: sogou spider
89Disallow: /
90
91# France
92User-agent: Exabot/3.0
93User-agent: Exalead
94User-agent: ExaLead Crawler
95Disallow: /
96
97# Vietnam
98User-agent: Wada.vn
99User-agent: Wada.vn Vietnamese Search
100User-agent: Wada.vn Vietnamese Search/2.1
101Disallow: /
102
103
104# South Korea
105User-agent: NaverBot
106User-agent: Yeti
107Disallow: /
108
109# Russia
110User-agent: Yandex
111User-agent: Yandex/1.01.001
112User-agent: YandexBot/3.0-MirrorDetector
113User-agent: YandexImages/3.0
114User-agent: YandexSomething/1.
115User-agent: Yandex.com
116User-agent: YandexBot/3.0
117User-agent: aiHitBot
118User-agent: aiHitBot/1.0
119User-agent: aiHitBot/1.1
120User-agent: Mail.RU_Bot/2.0
121User-agent: Mail.RU
122User-agent: Mail.Ru
123User-agent: Mail.RU_Bot/2.0; +http://go.mail.ru/help/robots
124Disallow: /
125
126# Japan
127User-agent: moget
128User-agent: ichiro
129User-agent: Ichiro
130User-agent: Ichiro 3.0
131User-agent: Arachmo
132User-agent: Zao
133Disallow: /
134
135# ALL BOTS
136User-agent: *
137Disallow: /atg/
138Disallow: /dyn/
139Disallow: /cart
140Disallow: /rest/
141Disallow: /checkout
142Disallow: /error/
143Disallow: /ymme/
144Disallow: /orderConfirmation
145Disallow: */user/
146Disallow: *recsPerPage*
147Disallow: */bn/
148Disallow: */bn$
149Disallow: */bn?
150Disallow: */under-2/
151Disallow: */no-rating/
152Disallow: */3-and-up/
153Disallow: */4-and-up/
154Disallow: */universal-products/b/
155Disallow: */vehiclespecific-products/b/
156Disallow: *?vehicleId=
157Disallow: *&vehicleId=
158Disallow: *?filters=
159Disallow: *&filters=
160Disallow: *?filterByKeyWord=
161Disallow: *&filterByKeyWord=
162Disallow: *?facet=
163Disallow: *&facet=
164Disallow: *?rrec=true
165Disallow: *&rrec=true
166Disallow: */_/N-
167Disallow: */US_CENSUS_NAME
168Disallow: *{US_CENSUS_NAME
169Disallow: /myzone/profile/login.jsp?redirectURL=
170Disallow: /signin*
171Allow: /signin$
172Disallow: /TealeafTarget
173Disallow: /errorPage
174Disallow: /ecomm/b2c/order/v2/availability
175Disallow: /ecomm/b2c/v1/store/preferredStore
176Disallow: /ecomm/b2c/v2/page/header
177Disallow: /ecomm/b2c/*/currentUser/
178Disallow: /diy/wp-admin/
179Disallow: */b/brand/*?facet=
180Disallow: /locations/es/
181Disallow: /locations/us/
182Disallow: *?bvstate=pg
183Disallow: *?searchText=
184Disallow: */diy/?s=
185
186# Block semantic search API endpoints
187Disallow: /sls/pd/product-navigation-search/v1/solr-search/autozone_diy_usa_concepts_live/semanticSearch*
188Disallow: /sls/pd/product-navigation-search/v1/solr-search/autozone_ims_concepts_live-mex/semanticSearch*
189
190Sitemap: https://www.autozone.com/sitemap_index.xml
191Sitemap: https://www.autozone.com/locations/sitemap.xml