Claude / robots.txt snapshot

← back to claude.ai · fetched 2026-07-30T16:59:37Z (6d ago) · HTTP 200 · 417 bytes · sha256 75ffa2efecdb519a · raw

final URL: https://claude.ai/robots.txt

1	User-Agent: ia_archiver
2	Allow: /$
3	Allow: /login
4	Disallow: /
5
6	User-Agent: GPTBot
7	User-Agent: OAI-SearchBot
8	User-Agent: ChatGPT-User
9	User-Agent: Google-Extended
10	Disallow: /
11
12	User-Agent: *
13	Disallow: /new?*
14	Disallow: /chat/*
15	Disallow: /join/*
16	Disallow: /magic-link*
17	Disallow: /api/*
18	Disallow: /onboarding*
19	Disallow: /upgrade*
20	Disallow: /lti/*
21	Disallow: /settings*
22	Disallow: /task*
23
24	Sitemap: https://claude.ai/sitemap.xml