NeuralCrawl

Hulu / robots.txt snapshot

← back to hulu.com · fetched 2026-07-08T02:36:30Z (29d ago) · HTTP 200 · 3111 bytes · sha256 8fd07f821d59f466 · raw

final URL: https://www.hulu.com/robots.txt

1# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file
2#robots.txt for www.hulu.com/
3
4User-agent: *
5Disallow: /
6
7# SEO Bots
8User-agent: Googlebot
9User-agent: Googlebot-Video
10User-agent: Googlebot-Image
11User-agent: Googlebot-News
12User-agent: Storebot-Google
13User-agent: Google-InspectionTool
14User-agent: Applebot
15User-agent: Bingbot
16User-agent: BingPreview
17User-agent: Slurp
18User-agent: DuckDuckBot
19User-agent: Baiduspider
20User-agent: Baiduspider-mobile
21User-agent: Baiduspider-video
22User-agent: Baiduspider-image
23User-agent: NaverBot
24User-agent: Y!J-WSC
25User-agent: YJ-WSC
26User-agent: Yeti
27User-agent: Yandex
28User-agent: YandexBot
29User-agent: YandexMobileBot
30User-agent: YandexVideo
31User-agent: YandexWebmaster
32User-agent: YandexSitelinks
33User-agent: SeznamBot
34User-agent: PetalBot
35User-agent: Sogou
36User-agent: Qwantify
37User-agent: Ecosia
38User-agent: 360Spider
39User-agent: Coc Coc
40User-agent: Qwantbot
41# SEO Tools
42User-agent: Botify
43User-agent: SemrushBot
44User-agent: Screaming Frog SEO Spider
45User-agent: DataForSeoBot
46User-agent: ia_archiver
47# Social
48User-agent: Facebot
49User-agent: Twitterbot
50User-agent: facebookexternalhit
51User-agent: instagram
52User-agent: Threads
53User-agent: TikTok
54User-agent: Pinterest
55User-agent: Pinterestbot
56User-agent: redditbot
57User-agent: WhatsApp
58User-agent: Snapchat
59User-agent: TelegramBot
60User-agent: Viber
61User-agent: KakaoTalk
62User-agent: Line
63# Ad Bots
64User-agent: AdsBot-Google
65User-agent: AdsBot-Google-Mobile
66User-agent: AdsBot-Google-Mobile-Apps
67User-agent: Mediapartners-Google
68User-agent: Google-Ads-Creatives-Assistant
69User-agent: FacebookAdPreview
70User-agent: meta-externalads
71User-agent: Twitterbot
72User-agent: Adidxbot
73User-agent: Bytespider
74User-agent: IAS-Crawler
75User-agent: DoubleVerifyBot
76User-agent: Moatbot
77User-agent: TheTradeDesk
78User-agent: TTDCrawler
79User-agent: CriteoBot
80# AI Bots
81User-agent: GPTBot
82User-agent: ChatGPT-User
83User-agent: OAI-SearchBot
84User-agent: Google-Extended
85User-agent: ClaudeBot
86User-agent: Claude-User
87User-agent: Claude-SearchBot
88User-agent: PerplexityBot
89User-agent: Applebot-Extended
90User-agent: Meta-ExternalAgent
91User-agent: Meta-ExternalFetcher
92User-agent: Amazonbot
93User-agent: CCBot
94User-agent: Google-Agent
95User-agent: GrokBot
96User-agent: YouBot
97User-agent: DuckAssistBot
98User-agent: DeepSeekBot
99Allow: /
100
101Disallow: /activities
102Disallow: /activity_feeds
103Disallow: /api
104Disallow: /browse
105Disallow: /channels
106Disallow: /discussions
107Disallow: /email
108Disallow: /exstack
109Disallow: /facebook_canvas
110Disallow: /feed
111Disallow: /highest_rated
112Disallow: /labs
113Disallow: /manifest.json
114Disallow: /masthead
115Disallow: /playlists
116Disallow: /plus
117Disallow: /popular
118Disallow: /posts
119Disallow: /profiles
120Disallow: /r
121Disallow: /recent
122Disallow: /released
123Disallow: /reviews
124Disallow: /rules
125Disallow: /search
126Disallow: /social
127Disallow: /start
128Disallow: /support
129Disallow: /*.swf
130Disallow: /tempo
131Disallow: /unsubscribe_email
132Disallow: /users
133Disallow: /videos
134Disallow: /watch
135Disallow: /widget
136
137
138# Announce Sitemap
139Sitemap: https://www.hulu.com/sitemap_index.xml
140
141# Source: site-static