Snyk / robots.txt snapshot
← back to snyk.io · fetched 2026-08-21T18:25:56Z (2mo ago) · HTTP 200 · 1019 bytes · sha256 d2ded0481ea1e4f1 · raw
final URL: https://snyk.io/robots.txt
| 1 | # * |
| 2 | User-agent: * |
| 3 | Allow: / |
| 4 | Content-Signal: ai-train=no, search=yes, ai-input=yes |
| 5 | |
| 6 | # * |
| 7 | User-agent: * |
| 8 | Disallow: /*.json$ |
| 9 | Disallow: /*_buildManifest.js$ |
| 10 | Disallow: /*_middlewareManifest.js$ |
| 11 | Disallow: /*_ssgManifest.js$ |
| 12 | Disallow: /*.js$ |
| 13 | |
| 14 | # * |
| 15 | User-agent: * |
| 16 | Disallow: /advisor |
| 17 | |
| 18 | # Host |
| 19 | Host: https://snyk.io |
| 20 | |
| 21 | # Sitemaps |
| 22 | Sitemap: https://snyk.io/sitemap-index.xml |
| 23 | Sitemap: https://snyk.io/sitemaps/sitemap-pages.xml |
| 24 | Sitemap: https://snyk.io/sitemaps/sitemap-articles.xml |
| 25 | Sitemap: https://snyk.io/sitemaps/sitemap-blogs.xml |
| 26 | Sitemap: https://snyk.io/sitemaps/sitemap-case-studies.xml |
| 27 | Sitemap: https://snyk.io/sitemaps/sitemap-contributors.xml |
| 28 | Sitemap: https://snyk.io/sitemaps/sitemap-events.xml |
| 29 | Sitemap: https://snyk.io/sitemaps/sitemap-podcasts.xml |
| 30 | Sitemap: https://snyk.io/sitemaps/sitemap-policies.xml |
| 31 | Sitemap: https://snyk.io/sitemaps/sitemap-press-releases.xml |
| 32 | Sitemap: https://snyk.io/sitemaps/sitemap-videos.xml |
| 33 | Sitemap: https://snyk.io/sitemaps/sitemap-misc.xml |
| 34 | Sitemap: https://snyk.io/advisor/sitemap/sitemap-index.xml |