GoDaddy / robots.txt
← back to godaddy.com · change detected 19d ago (2026-07-16T23:56:02Z) · modified +4 −0
What changed
- 🤖 AIAdded rules for AI crawler Claude-User (Anthropic)
Diff old (2026-06-20T01:10:31Z) → new (2026-07-16T23:56:02Z)
| @@ -1,6 +1,10 @@ | ||
| 1 | 1 | # |
| 2 | 2 | # robots.txt |
| 3 | 3 | # |
| 4 | + | |
| 5 | + User-agent: Claude-User | |
| 6 | + Allow: /domainsearch/ | |
| 7 | + Allow: /v1/search/ | |
| 4 | 8 | |
| 5 | 9 | User-agent: * |
| 6 | 10 | Allow: *.js |