NeuralCrawl

WhatsApp / robots.txt snapshot

← back to whatsapp.com · fetched 2026-11-05T19:53:47Z (1d ago) · HTTP 200 · 1226 bytes · sha256 ee97cafe79cc840a · raw

final URL: https://www.whatsapp.com/robots.txt

1# Notice: Collection of data on Facebook through automated means is
2# prohibited unless you have express written permission from Facebook
3# and may only be conducted for the limited purpose contained in said
4# permission.
5# See: http://www.facebook.com/apps/site_scraping_tos_terms.php
6
7User-agent: __usage_ai-training
8Disallow: /
9
10User-agent: Amazonbot
11Disallow: /
12
13User-agent: Applebot-Extended
14Disallow: /
15
16User-agent: Brightbot
17Disallow: /
18
19User-agent: ClaudeBot
20Disallow: /
21
22User-agent: Google-Extended
23Disallow: /
24
25User-agent: GPTBot
26Disallow: /
27
28User-agent: PerplexityBot
29Disallow: /
30
31User-agent: PetalBot
32Disallow: /
33
34User-agent: Scrapy
35Disallow: /
36
37User-agent: uptimerobot
38Disallow: /
39
40User-agent: viberbot
41Disallow: /
42
43User-agent: YaK
44Disallow: /
45
46User-agent: Yandex
47Disallow: /
48
49User-agent: Yeti
50Disallow: /
51
52User-agent: facebookexternalhit
53Allow: *
54
55User-agent: meta-externalads
56Allow: *
57
58User-agent: *
59Disallow: /*cursor=
60Disallow: /*fb_comment_id=
61Disallow: /ajax/
62Disallow: /tealium/
63Disallow: /intern/
64Disallow: /internal/
65Disallow: /login/
66Disallow: /oidc/callback/
67Disallow: /*.php
68Disallow: /android/third_party
69Disallow: /settings
70
71Sitemap: https://www.whatsapp.com/sitemap/www_whatsapp_com_sitemap.xml.gz