NeuralCrawl

Twitch / robots.txt snapshot

← back to twitch.tv · fetched 2026-11-05T01:38:00Z (2d ago) · HTTP 200 · 1674 bytes · sha256 c97fe4eb8c68471f · raw

final URL: https://www.twitch.tv/robots.txt

1User-agent: Amazonbot
2Disallow:
3
4User-agent: Amazonbot-Video
5Disallow:
6
7User-agent: Amzn-SearchBot
8Disallow:
9
10User-agent: Amzn-User
11Disallow:
12
13User-agent: *
14Disallow: /login$
15Disallow: /login?
16Disallow: /login/
17Disallow: /signup$
18Disallow: /signup?
19Disallow: /signup/
20Disallow: /auth/
21Disallow: /user/account-recovery
22Disallow: /user/password-reset
23Disallow: /user/not-me
24Disallow: /activate$
25Disallow: /activate?
26Disallow: /activate/
27Disallow: /admin/*
28Disallow: /wv/*
29Disallow: /user/*
30Disallow: /message/*
31Disallow: /settings/
32Disallow: /subscriptions$
33Disallow: /subscriptions?
34Disallow: /subscriptions/
35Disallow: /inventory$
36Disallow: /inventory?
37Disallow: /inventory/
38Disallow: /wallet$
39Disallow: /wallet?
40Disallow: /wallet/
41Disallow: /drops/
42Disallow: /payments/
43Disallow: /email-unsubscribe/
44Disallow: /search$
45Disallow: /search?
46Disallow: /search/
47Disallow: /bits-checkout
48Disallow: /redeem$
49Disallow: /redeem?
50Disallow: /redeem/
51Disallow: /claim$
52Disallow: /claim?
53Disallow: /claim/
54Disallow: /download-keys
55Disallow: /popout/
56Disallow: /embed/
57Disallow: /chat/
58Disallow: /moderator/
59Disallow: /stream$
60Disallow: /stream?
61Disallow: /stream/
62Disallow: /collab$
63Disallow: /collab?
64Disallow: /collab/
65Disallow: /guest-star
66Disallow: /sq$
67Disallow: /sq?
68Disallow: /sq/
69Disallow: /gl$
70Disallow: /gl?
71Disallow: /gl/
72Disallow: /sportradar-widgets
73Disallow: /beta/
74Disallow: /extensions/oauth-redirect
75Disallow: /payment-redirect
76Disallow: /copyright-claims
77Disallow: /report-appeal
78Disallow: /oss-attribution
79Disallow: /dj-signup
80Disallow: /r/
81Disallow: /stale/
82Allow: /directory
83Allow: /.well-known/assetlinks.json
84
85Sitemap: https://www.twitch.tv/sitemapv2_index.xml.gz