NeuralCrawl

JD.com / robots.txt snapshot

← back to jd.com · fetched 2026-06-21T13:32:37Z (5h ago) · HTTP 200 · 787 bytes · sha256 71185ce79642f0d9 · raw

final URL: https://jd.com/robots.txt

1User-agent: *
2Disallow: /*?
3
4User-agent: Googlebot
5Allow: /*?
6
7User-agent: Googlebot-Image
8Allow: /*?
9
10User-agent: Googlebot-Mobile
11Allow: /*?
12
13User-agent: Googlebot-Video
14Allow: /*?
15
16User-agent: Googlebot-News
17Allow: /*?
18
19User-agent: Bingbot
20Allow: /*?
21
22User-agent: msnbot
23Allow: /*?
24
25User-agent: bingbot-mobile
26Allow: /*?
27
28User-agent: Baiduspider
29Allow: /*?
30
31User-agent: Baiduspider-image
32Allow: /*?
33
34User-agent: Baiduspider-video
35Allow: /*?
36
37User-agent: YandexBot
38Allow: /*?
39
40User-agent: YandexBot-Image
41Allow: /*?
42
43User-agent: YandexBot-Mobile
44Allow: /*?
45
46User-agent: Sogou web spider
47Allow: /*?
48
49User-agent: Sogou Pic Spider
50Allow: /*?
51
52User-agent: 360Spider
53Allow: /*?
54
55User-agent: HaosouSpider
56Allow: /*?
57
58User-agent: Sosoimagespider
59Allow: /*?
60
61User-agent: Sosospider
62Allow: /*?