NeuralCrawl

Snapchat / robots.txt snapshot

← back to snapchat.com · fetched 2026-06-20T01:10:30Z (18h ago) · HTTP 200 · 4304 bytes · sha256 34ad1901121a685f · raw

final URL: https://www.snapchat.com/robots.txt

1# *
2User-agent: *
3Disallow: /verificationofemployment
4Disallow: /static_files/deletion_request.pdf
5Disallow: /static/gla/
6Disallow: /invite/
7Disallow: /local/
8
9# Meta-WebIndexer
10User-agent: Meta-WebIndexer
11Allow: /
12Disallow: /local/
13
14# Meta-ExternalAgent
15User-agent: Meta-ExternalAgent
16Disallow: /
17
18# Meta-ExternalFetcher
19User-agent: Meta-ExternalFetcher
20Disallow: /
21
22# AI2Bot
23User-agent: AI2Bot
24Disallow: /
25
26# Ai2Bot-Dolma
27User-agent: Ai2Bot-Dolma
28Disallow: /
29
30# Amazonbot
31User-agent: Amazonbot
32Disallow: /
33
34# anthropic-ai
35User-agent: anthropic-ai
36Disallow: /
37
38# Applebot-Extended
39User-agent: Applebot-Extended
40Disallow: /
41
42# Brightbot 1.0
43User-agent: Brightbot 1.0
44Disallow: /
45
46# Bytespider
47User-agent: Bytespider
48Disallow: /
49
50# CCBot
51User-agent: CCBot
52Disallow: /
53
54# ChatGPT-User
55User-agent: ChatGPT-User
56Disallow: /
57
58# Claude-Web
59User-agent: Claude-Web
60Disallow: /
61
62# ClaudeBot
63User-agent: ClaudeBot
64Disallow: /
65
66# cohere-ai
67User-agent: cohere-ai
68Disallow: /
69
70# cohere-training-data-crawler
71User-agent: cohere-training-data-crawler
72Disallow: /
73
74# Crawlspace
75User-agent: Crawlspace
76Disallow: /
77
78# Diffbot
79User-agent: Diffbot
80Disallow: /
81
82# DuckAssistBot
83User-agent: DuckAssistBot
84Disallow: /
85
86# FacebookBot
87User-agent: FacebookBot
88Disallow: /
89
90# FriendlyCrawler
91User-agent: FriendlyCrawler
92Disallow: /
93
94# Googlebot
95User-agent: Googlebot
96Disallow: /getreplies
97Disallow: /explore/
98Disallow: /local/
99
100# Google-Extended
101User-agent: Google-Extended
102Disallow: /
103
104# GoogleOther
105User-agent: GoogleOther
106Disallow: /
107
108# GoogleOther-Image
109User-agent: GoogleOther-Image
110Disallow: /
111
112# GoogleOther-Video
113User-agent: GoogleOther-Video
114Disallow: /
115
116# GPTBot
117User-agent: GPTBot
118Disallow: /
119
120# iaskspider/2.0
121User-agent: iaskspider/2.0
122Disallow: /
123
124# ICC-Crawler
125User-agent: ICC-Crawler
126Disallow: /
127
128# ImagesiftBot
129User-agent: ImagesiftBot
130Disallow: /
131
132# img2dataset
133User-agent: img2dataset
134Disallow: /
135
136# ISSCyberRiskCrawler
137User-agent: ISSCyberRiskCrawler
138Disallow: /
139
140# Kangaroo Bot
141User-agent: Kangaroo Bot
142Disallow: /
143
144# OAI-SearchBot
145User-agent: OAI-SearchBot
146Disallow: /
147
148# omgili
149User-agent: omgili
150Disallow: /
151
152# omgilibot
153User-agent: omgilibot
154Disallow: /
155
156# PanguBot
157User-agent: PanguBot
158Disallow: /
159
160# PerplexityBot
161User-agent: PerplexityBot
162Disallow: /
163
164# Perplexity‑User
165User-agent: Perplexity‑User
166Disallow: /
167
168# PetalBot
169User-agent: PetalBot
170Disallow: /
171
172# Scrapy
173User-agent: Scrapy
174Disallow: /
175
176# SemrushBot-OCOB
177User-agent: SemrushBot-OCOB
178Disallow: /
179
180# SemrushBot-SWA
181User-agent: SemrushBot-SWA
182Disallow: /
183
184# Sidetrade indexer bot
185User-agent: Sidetrade indexer bot
186Disallow: /
187
188# Timpibot
189User-agent: Timpibot
190Disallow: /
191
192# VelenPublicWebCrawler
193User-agent: VelenPublicWebCrawler
194Disallow: /
195
196# Webzio-Extended
197User-agent: Webzio-Extended
198Disallow: /
199
200# YouBot
201User-agent: YouBot
202Disallow: /
203
204# Host
205Host: https://www.snapchat.com
206
207# Sitemaps
208Sitemap: https://www.snapchat.com/sitemap.xml
209Sitemap: https://www.snapchat.com/lens/sitemap.xml
210Sitemap: https://www.snapchat.com/add/sitemap.xml
211Sitemap: https://www.snapchat.com/p/sitemap.xml
212Sitemap: https://www.snapchat.com/sitemaps/place/sitemap-index.xml
213Sitemap: https://www.snapchat.com/sitemaps/add/sitemap-index.xml
214Sitemap: https://www.snapchat.com/sitemaps/p/sitemap-index.xml
215Sitemap: https://www.snapchat.com/sitemaps/tag/sitemap-index.xml
216Sitemap: https://www.snapchat.com/sitemaps/topic/sitemap-index.xml
217Sitemap: https://www.snapchat.com/sitemaps/curated_topic/sitemap-index.xml
218Sitemap: https://www.snapchat.com/sitemaps/extended_curated_topic/sitemap-index.xml
219Sitemap: https://www.snapchat.com/sitemaps/ddb_topic_all/sitemap-index.xml
220Sitemap: https://www.snapchat.com/sitemaps/ddb_topic_eight_plus/sitemap-index.xml
221Sitemap: https://www.snapchat.com/sitemaps/ddb_topic_up_to_eight/sitemap-index.xml
222Sitemap: https://www.snapchat.com/sitemaps/top_spotlights/sitemap-index.xml
223Sitemap: https://www.snapchat.com/sitemaps/mit_tier1_legacy/sitemap-index.xml
224Sitemap: https://www.snapchat.com/sitemaps/mit_tier1_new/sitemap-index.xml
225Sitemap: https://www.snapchat.com/sitemaps/mit_tier1_mix/sitemap-index.xml
226Sitemap: https://www.snapchat.com/sitemaps/mit_tier2_legacy/sitemap-index.xml
227Sitemap: https://www.snapchat.com/sitemaps/mit_tier2_new/sitemap-index.xml
228Sitemap: https://www.snapchat.com/sitemaps/mit_tier2_mix/sitemap-index.xml