Arkose Labs / robots.txt
← back to arkoselabs.com · change detected 2mo ago (2026-08-27T02:04:19Z) · modified +1 −2
What changed
- Blocked user-agent ‘SiteAuditBot’ entirely
- Removed Disallow: (empty) for SiteAuditBot
Diff old (2026-07-13T16:47:36Z) → new (2026-08-27T02:04:19Z)
| @@ -1,5 +1,4 @@ | ||
| 1 | 1 | User-agent: * |
| 2 | - Content-Signal: ai-train=yes, search=yes, ai-input=yes | |
| 3 | 2 | Allow: / |
| 4 | 3 | Disallow: /search-results |
| 5 | 4 | Disallow: /404 |
| @@ -8,6 +7,6 @@ | ||
| 8 | 7 | Disallow: / |
| 9 | 8 | |
| 10 | 9 | User-agent: SiteAuditBot |
| 11 | - Disallow: | |
| 10 | + Disallow: / | |
| 12 | 11 | |
| 13 | 12 | Sitemap: https://www.arkoselabs.com/sitemap.xml |