NeuralCrawl

Walmart / robots.txt snapshot

← back to walmart.com · fetched 2026-06-20T01:10:30Z (18h ago) · HTTP 200 · 3580 bytes · sha256 ff8182a46d9d37d2 · raw

final URL: https://www.walmart.com/robots.txt

1#Sitemaps-https
2
3Sitemap: https://www.walmart.com/sitemap_category.xml
4Sitemap: https://www.walmart.com/sitemap_store_main.xml
5
6Sitemap: https://www.walmart.com/help/sitemap_gm.xml
7Sitemap: https://www.walmart.com/sitemap_browse_fst.xml
8Sitemap: https://www.walmart.com/sitemap_store_dept.xml
9
10Sitemap: https://www.walmart.com/sitemap_tp_legacy.xml
11Sitemap: https://www.walmart.com/sitemap_tp_at.xml
12Sitemap: https://www.walmart.com/sitemap_tp_br.xml
13Sitemap: https://www.walmart.com/sitemap_tp_kp.xml
14
15Sitemap: https://www.walmart.com/sitemap_brp_02.xml
16Sitemap: https://www.walmart.com/sitemap_brp_03.xml
17Sitemap: https://www.walmart.com/sitemap_brp_04.xml
18Sitemap: https://www.walmart.com/sitemap_brp_05.xml
19Sitemap: https://www.walmart.com/sitemap_brp_06.xml
20Sitemap: https://www.walmart.com/sitemap_brp_07.xml
21Sitemap: https://www.walmart.com/sitemap_brp_08.xml
22Sitemap: https://www.walmart.com/sitemap_brp_09.xml
23
24Sitemap: https://www.walmart.com/sitemap_itp_01.xml
25Sitemap: https://www.walmart.com/sitemap_itp_02.xml
26Sitemap: https://www.walmart.com/sitemap_itp_03.xml
27
28Sitemap: https://www.walmart.com/sitemap_rp_01.xml
29Sitemap: https://www.walmart.com/sitemap_rp_02.xml
30Sitemap: https://www.walmart.com/sitemap_rp_03.xml
31Sitemap: https://www.walmart.com/sitemap_rp_04.xml
32Sitemap: https://www.walmart.com/sitemap_rp_05.xml
33
34Sitemap: https://www.walmart.com/sitemap_slp_01.xml
35Sitemap: https://www.walmart.com/sitemap_slp_02.xml
36
37Sitemap: https://www.walmart.com/sitemap_auto_dir.xml
38
39Sitemap: https://www.walmart.com/sitemap_brndshp_brand.xml
40
41Sitemap: https://www.walmart.com/sitemap_sp.xml
42
43Sitemap: https://www.walmart.com/sitemap_hi_ip.xml
44Sitemap: https://www.walmart.com/sitemap_hi_tp.xml
45Sitemap: https://www.walmart.com/sitemap_hi_bp.xml
46Sitemap: https://www.walmart.com/sitemap_hi_mp.xml
47
48
49#Disallow select URLs
50User-agent: *
51Disallow: /0/
52Disallow: /55875582/walmart-us/catalog/
53Disallow: /account/
54Disallow: /api/
55Disallow: /collection/api/logger
56Disallow: /cp/-201
57Disallow: /cp/-302
58Disallow: /cp/-306
59Disallow: /cp/-309
60Disallow: /cp/-506
61Disallow: /cp/-509
62Disallow: /cp/api/logger
63Disallow: /cp/api/wpa
64Disallow: /cservice/
65Disallow: /cservice/ya_index.gsp
66Disallow: /electrode/api/logger
67Disallow: /email_collect/
68Disallow: /giftregistry/
69Disallow: /msp
70Disallow: /nonConfig/api/wpa
71Disallow: /popup_security.jsp
72Disallow: /product/electrode/api/logger
73Disallow: /product/electrode/api/wpa
74Allow: /reviews/product/
75Allow: /reviews/seller/
76Disallow: /search
77Disallow: /search/api/wpa
78Disallow: */api/wpa
79Disallow: */midas/*
80Disallow: /search/search-ng.do
81Disallow: /solutions/
82Disallow: /store/ajax/detail-navigation
83Disallow: /store/ajax/local-store
84Disallow: /store/ajax/preferred
85Disallow: /store/ajax/search
86Disallow: /store/ajax/visited-store
87Disallow: /store/category/
88Disallow: /store/electrode/api/fetch-coupons
89Disallow: /store/electrode/api/logger
90Disallow: /store/electrode/api/p13n
91Disallow: /store/electrode/api/search
92Disallow: /store/electrode/api/stores
93Allow: /store/finder
94Disallow: /store/popular_in_grade/
95Disallow: /storeLocator/
96Disallow: /storeLocator/ca_storefinder_results.do
97Disallow: /tealeaf
98Disallow: /topic/electrode/api/logger
99Disallow: /topic/electrode/api/wpa
100Disallow: /typeahead/
101Disallow: /wmflows/
102Disallow: */undefined/*
103Disallow: /store/*/search
104Disallow: /c/kp/*/page/*
105Disallow: /feeds/*
106Disallow: /orders
107Disallow: /es/
108Disallow: /swag/*
109Disallow: /dad/trk
110Disallow: /sp/*
111
112#Crawler specific settings
113User-agent: Adsbot-Google
114
115User-agent: Mediapartners-Google
116
117# slow down Yahoo
118User-agent: Slurp
119Crawl-delay: 5