Claude / robots.txt
← back to claude.ai · change detected 2mo ago (2026-09-04T16:59:44Z) · modified +1 −8
What changed
- Added Allow: /api/desktop/ for *
- Removed Disallow: /chat/* for *
- Removed Disallow: /join/* for *
- Removed Disallow: /magic-link* for *
- Removed Disallow: /new?* for *
- Removed Disallow: /onboarding* for *
- Removed Disallow: /settings* for *
- Removed Disallow: /task* for *
- Removed Disallow: /upgrade* for *
Diff old (2026-07-30T16:59:37Z) → new (2026-09-04T16:59:44Z)
| @@ -10,15 +10,8 @@ | ||
| 10 | 10 | Disallow: / |
| 11 | 11 | |
| 12 | 12 | User-Agent: * |
| 13 | - Disallow: /new?* | |
| 14 | - Disallow: /chat/* | |
| 15 | - Disallow: /join/* | |
| 16 | - Disallow: /magic-link* | |
| 13 | + Allow: /api/desktop/ | |
| 17 | 14 | Disallow: /api/* |
| 18 | - Disallow: /onboarding* | |
| 19 | - Disallow: /upgrade* | |
| 20 | 15 | Disallow: /lti/* |
| 21 | - Disallow: /settings* | |
| 22 | - Disallow: /task* | |
| 23 | 16 | |
| 24 | 17 | Sitemap: https://claude.ai/sitemap.xml |