NeuralCrawl

Snapchat / robots.txt snapshot

← back to snapchat.com · fetched 2026-07-10T03:49:59Z (8d ago) · HTTP 200 · 3926 bytes · sha256 a1c6e7ce38ca4498 · raw

final URL: https://www.snapchat.com/robots.txt

1# *
2User-agent: *
3Disallow: /verificationofemployment
4Disallow: /static_files/deletion_request.pdf
5Disallow: /static/gla/
6Disallow: /invite/
7Disallow: /local/
8Disallow: /entertainment/
9
10# Meta-WebIndexer
11User-agent: Meta-WebIndexer
12Allow: /
13Disallow: /local/
14Disallow: /entertainment/
15
16# Meta-ExternalAgent
17User-agent: Meta-ExternalAgent
18Disallow: /
19
20# Meta-ExternalFetcher
21User-agent: Meta-ExternalFetcher
22Disallow: /
23
24# AI2Bot
25User-agent: AI2Bot
26Disallow: /
27
28# Ai2Bot-Dolma
29User-agent: Ai2Bot-Dolma
30Disallow: /
31
32# Amazonbot
33User-agent: Amazonbot
34Disallow: /
35
36# anthropic-ai
37User-agent: anthropic-ai
38Disallow: /
39
40# Applebot-Extended
41User-agent: Applebot-Extended
42Disallow: /
43
44# Brightbot 1.0
45User-agent: Brightbot 1.0
46Disallow: /
47
48# Bytespider
49User-agent: Bytespider
50Disallow: /
51
52# CCBot
53User-agent: CCBot
54Disallow: /
55
56# ChatGPT-User
57User-agent: ChatGPT-User
58Disallow: /
59
60# Claude-Web
61User-agent: Claude-Web
62Disallow: /
63
64# ClaudeBot
65User-agent: ClaudeBot
66Disallow: /
67
68# cohere-ai
69User-agent: cohere-ai
70Disallow: /
71
72# cohere-training-data-crawler
73User-agent: cohere-training-data-crawler
74Disallow: /
75
76# Crawlspace
77User-agent: Crawlspace
78Disallow: /
79
80# Diffbot
81User-agent: Diffbot
82Disallow: /
83
84# DuckAssistBot
85User-agent: DuckAssistBot
86Disallow: /
87
88# FacebookBot
89User-agent: FacebookBot
90Disallow: /
91
92# FriendlyCrawler
93User-agent: FriendlyCrawler
94Disallow: /
95
96# Googlebot
97User-agent: Googlebot
98Disallow: /getreplies
99Disallow: /explore/
100Disallow: /local/
101Disallow: /entertainment/
102
103# Google-Extended
104User-agent: Google-Extended
105Disallow: /
106
107# GoogleOther
108User-agent: GoogleOther
109Disallow: /
110
111# GoogleOther-Image
112User-agent: GoogleOther-Image
113Disallow: /
114
115# GoogleOther-Video
116User-agent: GoogleOther-Video
117Disallow: /
118
119# GPTBot
120User-agent: GPTBot
121Disallow: /
122
123# iaskspider/2.0
124User-agent: iaskspider/2.0
125Disallow: /
126
127# ICC-Crawler
128User-agent: ICC-Crawler
129Disallow: /
130
131# ImagesiftBot
132User-agent: ImagesiftBot
133Disallow: /
134
135# img2dataset
136User-agent: img2dataset
137Disallow: /
138
139# ISSCyberRiskCrawler
140User-agent: ISSCyberRiskCrawler
141Disallow: /
142
143# Kangaroo Bot
144User-agent: Kangaroo Bot
145Disallow: /
146
147# OAI-SearchBot
148User-agent: OAI-SearchBot
149Disallow: /
150
151# omgili
152User-agent: omgili
153Disallow: /
154
155# omgilibot
156User-agent: omgilibot
157Disallow: /
158
159# PanguBot
160User-agent: PanguBot
161Disallow: /
162
163# PerplexityBot
164User-agent: PerplexityBot
165Disallow: /
166
167# Perplexity‑User
168User-agent: Perplexity‑User
169Disallow: /
170
171# PetalBot
172User-agent: PetalBot
173Disallow: /
174
175# Scrapy
176User-agent: Scrapy
177Disallow: /
178
179# SemrushBot-OCOB
180User-agent: SemrushBot-OCOB
181Disallow: /
182
183# SemrushBot-SWA
184User-agent: SemrushBot-SWA
185Disallow: /
186
187# Sidetrade indexer bot
188User-agent: Sidetrade indexer bot
189Disallow: /
190
191# Timpibot
192User-agent: Timpibot
193Disallow: /
194
195# VelenPublicWebCrawler
196User-agent: VelenPublicWebCrawler
197Disallow: /
198
199# Webzio-Extended
200User-agent: Webzio-Extended
201Disallow: /
202
203# YouBot
204User-agent: YouBot
205Disallow: /
206
207# Host
208Host: https://www.snapchat.com
209
210# Sitemaps
211Sitemap: https://www.snapchat.com/sitemap.xml
212Sitemap: https://www.snapchat.com/lens/sitemap.xml
213Sitemap: https://www.snapchat.com/add/sitemap.xml
214Sitemap: https://www.snapchat.com/p/sitemap.xml
215Sitemap: https://www.snapchat.com/sitemaps/place/sitemap-index.xml
216Sitemap: https://www.snapchat.com/sitemaps/add/sitemap-index.xml
217Sitemap: https://www.snapchat.com/sitemaps/p/sitemap-index.xml
218Sitemap: https://www.snapchat.com/sitemaps/tag/sitemap-index.xml
219Sitemap: https://www.snapchat.com/sitemaps/topic/sitemap-index.xml
220Sitemap: https://www.snapchat.com/sitemaps/curated_topic/sitemap-index.xml
221Sitemap: https://www.snapchat.com/sitemaps/extended_curated_topic/sitemap-index.xml
222Sitemap: https://www.snapchat.com/sitemaps/ddb_topic_all/sitemap-index.xml
223Sitemap: https://www.snapchat.com/sitemaps/ddb_topic_eight_plus/sitemap-index.xml
224Sitemap: https://www.snapchat.com/sitemaps/ddb_topic_up_to_eight/sitemap-index.xml
225Sitemap: https://www.snapchat.com/sitemaps/top_spotlights/sitemap-index.xml