🇺🇸 Dropbox

dropbox.com · Top 1000 websites · rank #110 · Web · live robots.txt ↗

AI crawler access (latest snapshot, 1h ago)

⛔blocked restricted ✅allowed faded = inherited from the * wildcard group

✅GPTBot

✅ChatGPT-User

✅OAI-SearchBot

✅ClaudeBot

✅Claude-User

✅Claude-SearchBot

✅anthropic-ai

✅Claude-Web

✅CCBot

✅Google-Extended

✅Applebot-Extended

✅PerplexityBot

✅Perplexity-User

✅Bytespider

✅Amazonbot

✅FacebookBot

✅meta-externalagent

✅meta-externalfetcher

✅cohere-ai

✅AI2Bot

✅Diffbot

✅omgili

✅YouBot

✅DuckAssistBot

✅MistralAI-User

✅PanguBot

✅Timpibot

Current robots.txt 830 bytes · sha256 2ea50072fe93 · raw

User-agent: Twitterbot
Disallow:

User-agent: facebookexternalhit
Allow: /s/
Allow: /sh/
Allow: /scl/
Allow: /sc/

User-agent: *
Allow: /static/css/
Allow: /static/images/
Allow: /static/js/
Allow: /llms.txt
Disallow: /static/
Disallow: /password_expired
Disallow: /help/sl
Disallow: /help/vodafone
Disallow: /home*
Disallow: /package_files_uploaded_via_chrome
Disallow: /photos/app_store_link/
Disallow: /recover*
Disallow: /sharing/*
Disallow: /sharing/folders*
Disallow: /sharing/files*
Disallow: /requests*
Disallow: /account*
Disallow: /referrals*
Disallow: /downgrade*
Disallow: /developers-v1
Disallow: /developers/downloads/sdks/
Disallow: /developers/whats_new
Disallow: /reengageoptout/*
Disallow: /preview/
Disallow: /s/
Disallow: /sh/
Disallow: /scl/
Disallow: /sc/


Sitemap: https://www.dropbox.com/sitemapindex.xml

Change history

initial 1h ago snapshot
- First snapshot of robots.txt archived