xAI / robots.txt snapshot
← back to x.ai · fetched 2026-09-15T02:23:28Z (4d ago) · HTTP 200 · 223 bytes · sha256 069241f196e6caa2 · raw
final URL: https://x.ai/robots.txt
| 1 | User-agent: * |
| 2 | Allow: / |
| 3 | Disallow: /tools/ |
| 4 | |
| 5 | Sitemap: https://x.ai/sitemap.xml |
| 6 | |
| 7 | # Content Signals — declare AI/search usage preferences |
| 8 | # See: https://contentsignals.org/ |
| 9 | Content-Signal: ai-train=yes, search=yes, ai-input=yes |