🇺🇸 Imperva
imperva.com · Cybersecurity · rank #49 · Cybersecurity · live robots.txt ↗
AI crawler access (latest snapshot, 5h ago)
⛔blocked
restricted
✅allowed
faded = inherited from the * wildcard group
✅GPTBot
✅ChatGPT-User
✅OAI-SearchBot
✅ClaudeBot
✅Claude-User
✅Claude-SearchBot
✅anthropic-ai
✅Claude-Web
✅CCBot
✅Google-Extended
✅Applebot-Extended
✅PerplexityBot
✅Perplexity-User
✅Bytespider
✅Amazonbot
✅FacebookBot
✅meta-externalagent
✅meta-externalfetcher
✅cohere-ai
✅AI2Bot
✅Diffbot
✅omgili
✅YouBot
✅DuckAssistBot
✅MistralAI-User
✅PanguBot
✅Timpibot
Current robots.txt 2419 bytes · sha256 a1eab49b1a10 · raw
User-agent: * Disallow: /docs/channel/ Disallow: /?hl=cert_ssl Disallow: /cred/login.php Disallow: /finance Disallow: /private Disallow: /cred/password Disallow: /cred/profiles Disallow: /*sa=3&msg=1 Disallow: /docs/gated/ Disallow: /download.asp* Disallow: /resources/resource-library/webinars/* Disallow: /blog/tag/ Allow: /resources/resource-library/webinars/$ Disallow: /mangedservices-resources Disallow: */blog/*/feed/ Disallow: /why-am-i-seeing-this-page/ Disallow: /other/ Disallow: /resources/reports/Imperva_TheStateOfVulnerabilitiesIn2020_Report-Final.pdf Disallow: /legal/wp-content/uploads/sites/14/2021/11/IPA-and-DA-for-Form.pdf User-agent: Adsbot-Google Disallow: /*sa=3&msg=1 Disallow: /docs/gated/ Disallow: /download.asp* User-agent: WBSearchBot Disallow: / Disallow: /docs/channel/ Disallow: /docs/gated/ Disallow: /download.asp* User-agent: magpie-crawler Disallow: / Disallow: /docs/channel/ Disallow: /docs/gated/ Disallow: /download.asp* User-agent: Twitterbot Disallow: /docs/channel/ Disallow: /docs/gated/ Disallow: /download.asp* # START YOAST BLOCK # --------------------------- User-agent: * Disallow: Sitemap: https://www.imperva.com/sitemap_index.xml Sitemap: https://www.imperva.com/resources/sitemap_index.xml Sitemap: https://www.imperva.com/solutions/sitemap_index.xml Sitemap: https://www.imperva.com/blog/sitemap_index.xml Sitemap: https://www.imperva.com/partners/sitemap_index.xml Sitemap: https://www.imperva.com/products/sitemap_index.xml Sitemap: https://www.imperva.com/company/sitemap_index.xml Sitemap: https://www.imperva.com/learn/sitemap_index.xml Sitemap: https://www.imperva.com/legal/sitemap_index.xml Sitemap: https://www.imperva.com/cyber-threat-index/sitemap_index.xml Sitemap: https://www.imperva.com/mangedservices-resources/sitemap_index.xml Sitemap: https://www.imperva.com/data-security-coverage-tool/sitemap_index.xml Sitemap: https://www.imperva.com/cyber-threat-attack-map/sitemap_index.xml Sitemap: https://www.imperva.com/cloud-security/sitemap_index.xml Sitemap: https://www.imperva.com/hackathon/sitemap_index.xml Sitemap: https://www.imperva.com/support/sitemap_index.xml Sitemap: https://www.imperva.com/trust-center/sitemap_index.xml Sitemap: https://www.imperva.com/fr/sitemap_index.xml Sitemap: https://www.imperva.com/kr/sitemap_index.xml Sitemap: https://www.imperva.com/th/sitemap_index.xml # --------------------------- # END YOAST BLOCK
Change history
-
initial snapshot
- First snapshot of robots.txt archived