NeuralCrawl

Discord / robots.txt snapshot

← back to discord.com · fetched 2026-06-20T01:10:30Z (15h ago) · HTTP 200 · 1717 bytes · sha256 3b8820b34412f5f2 · raw

final URL: https://discord.com/robots.txt

1User-agent: *
2Allow: /api/v*/store
3Allow: /api/v*/applications
4Allow: /api/v*/application-directory-static
5Allow: /api/v*/invite
6Allow: /api/v*/discovery
7Allow: /api/discovery
8Allow: /invite
9Allow: /invite/
10Allow: /terms
11Allow: /privacy
12Disallow: /channels
13Disallow: /channels/
14Disallow: /verify
15Disallow: /verify/
16Disallow: /reset
17Disallow: /reset/
18Disallow: /authorize-ip
19Disallow: /authorize-ip/
20Disallow: /reject-ip
21Disallow: /reject-ip/
22Disallow: /reject-mfa
23Disallow: /reject-mfa/
24Disallow: /oauth2
25Disallow: /oauth2/
26Disallow: /api
27Disallow: /api/
28Disallow: /widget
29Disallow: /widget/
30Disallow: /profile/
31Disallow: /report-review
32Disallow: /report-review/
33Disallow: /store/skus/
34Disallow: /wasntme
35Disallow: /wasntme/
36
37User-agent: Twitterbot
38Disallow: /channels
39Disallow: /channels/
40Disallow: /verify
41Disallow: /verify/
42Disallow: /reset
43Disallow: /reset/
44Disallow: /authorize-ip
45Disallow: /authorize-ip/
46Disallow: /reject-ip
47Disallow: /reject-ip/
48Disallow: /reject-mfa
49Disallow: /reject-mfa/
50Disallow: /report-review
51Disallow: /report-review/
52Disallow: /oauth2
53Disallow: /oauth2/
54Disallow: /api
55Disallow: /api/
56Disallow: /widget
57Disallow: /widget/
58Disallow: /profile/
59Allow: /api/v*/application-directory-static
60Allow: /invite
61Allow: /invite/
62Allow: /terms
63Allow: /privacy
64
65User-agent: nsa
66Disallow: /
67
68Sitemap: https://discord.com/servers/servers-sitemap-index.xml
69Sitemap: https://discord.com/servers/servers-parent-search-sitemap-index.xml
70Sitemap: https://discord.com/application-directory-seo/sitemap-index.xml
71Sitemap: https://discord.com/announcements/announcements-sitemap-index.xml
72Sitemap: https://discord.com/trending-games/trending-games-sitemap-index.xml
73Sitemap: https://discord.com/blog/rss.xml