Snyk / robots.txt
← back to snyk.io · change detected 15d ago (2026-10-22T21:05:52Z) · modified +4 −15
What changed
- Removed Disallow: /*.json$ for *
- Removed Disallow: /*_buildManifest.js$ for *
- Removed Disallow: /*_middlewareManifest.js$ for *
- Removed Disallow: /*_ssgManifest.js$ for *
Diff old (2026-09-09T00:14:16Z) → new (2026-10-22T21:05:52Z)
| @@ -1,29 +1,18 @@ | ||
| 1 | - # * | |
| 2 | - User-agent: * | |
| 1 | + User-Agent: * | |
| 3 | 2 | Allow: / |
| 4 | 3 | Content-Signal: ai-train=no, search=yes, ai-input=yes |
| 5 | 4 | |
| 6 | - # * | |
| 7 | - User-agent: * | |
| 8 | - Disallow: /*.json$ | |
| 9 | - Disallow: /*_buildManifest.js$ | |
| 10 | - Disallow: /*_middlewareManifest.js$ | |
| 11 | - Disallow: /*_ssgManifest.js$ | |
| 5 | + User-Agent: * | |
| 12 | 6 | Disallow: /*.js$ |
| 13 | 7 | |
| 14 | - # * | |
| 15 | - User-agent: * | |
| 8 | + User-Agent: * | |
| 16 | 9 | Disallow: /advisor |
| 17 | 10 | |
| 18 | - # * | |
| 19 | - User-agent: * | |
| 11 | + User-Agent: * | |
| 20 | 12 | Disallow: /*?topic= |
| 21 | 13 | Disallow: /*&topic= |
| 22 | 14 | |
| 23 | - # Host | |
| 24 | 15 | Host: https://snyk.io |
| 25 | - | |
| 26 | - # Sitemaps | |
| 27 | 16 | Sitemap: https://snyk.io/sitemap-index.xml |
| 28 | 17 | Sitemap: https://snyk.io/sitemaps/sitemap-pages.xml |
| 29 | 18 | Sitemap: https://snyk.io/sitemaps/sitemap-articles.xml |