Claude / robots.txt snapshot

← back to claude.ai · fetched 2026-06-26T13:23:15Z (3h ago) · HTTP 200 · 408 bytes · sha256 7edaaba79d5dc108 · raw

final URL: https://claude.ai/robots.txt

1	User-Agent: ia_archiver
2	Allow: /$
3	Allow: /login
4	Disallow: /
5
6	User-Agent: GPTBot
7	User-Agent: OAI-SearchBot
8	User-Agent: ChatGPT-User
9	Disallow: /
10
11	User-Agent: *
12	Disallow: /new?*
13	Disallow: /chat/*
14	Disallow: /share/*
15	Disallow: /join/*
16	Disallow: /magic-link*
17	Disallow: /api/*
18	Disallow: /onboarding*
19	Disallow: /upgrade*
20	Disallow: /lti/*
21	Disallow: /settings*
22	Disallow: /task*
23
24	Sitemap: https://claude.ai/sitemap.xml