NeuralCrawl

πŸ‡ΊπŸ‡Έ Splunk

splunk.com · Cybersecurity · rank #12 · Cybersecurity · live robots.txt ↗

AI crawler access (latest snapshot, 5h ago)

blocked restricted allowed faded = inherited from the * wildcard group

GPTBot
ChatGPT-User
OAI-SearchBot
ClaudeBot
Claude-User
Claude-SearchBot
anthropic-ai
Claude-Web
CCBot
Google-Extended
Applebot-Extended
PerplexityBot
Perplexity-User
Bytespider
Amazonbot
FacebookBot
meta-externalagent
meta-externalfetcher
cohere-ai
AI2Bot
Diffbot
omgili
YouBot
DuckAssistBot
MistralAI-User
PanguBot
Timpibot

Current robots.txt 7640 bytes · sha256 a43bd32b12f3 · raw

# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWKXWMMMMMMMMMMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWWNXkc'cXMMMMMMMMMMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWKkoc:;..'..codONWMWNWMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMW0d;'';:,..;,.'lc;;:oxl,xWMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWO:..,:cl:..,;'.'cllolc,..oWMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWKc..,loc:c;...',..:clodxdl,:0WMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWO,.,::ll;'.....','.,lddddxxo,;OWMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWk'.;lc;;,...','''''..,:ooc:cllckWMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM0,.;llc'..',,,,''''''''',',xKNWMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMNc.,cc:'.',,,,,'....,:;,,ll,cKWMMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMk..:c;..,,,,',,'.  .:ldc,:kd';kXWMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWl.':;..',,,'.,,,.  .':kd;,o0l.oNWMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMX:.,:'.',,,,'..,,,'...,:;,,:Ok'lNMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMWWMMMMMMMMMMMMMMMMMMMMMMMMMMMMK;.,'.',,,,,....',,,,,,,,,,,::',lxKWMMMMMMMMMMMMMMMM
# MMMMWWMMMMMMMMMMMMMMMNX0xolcccclox0NMMMMMMMMMMMMMMMMMMMMMMK;.'..,;,,,''''......'',;,,,;;;::,;kWMMMMMMMMMMMMMMM
# MMMKlcx0NWWWWNNX0Odoc:;;;;:clllc;,';dXWMMMMMMMMMMMMMMMMMMMX:.'.';,,,,'',,'...:l,.;:,';:'.,:c';KMMMMMMMMMMMMMMM
# MMWl...,;:::;;,,,,,;:looddddddddxxo;.,OWMMMMMMMMMMMMMMMMMMWl...;;,,,,',,,,,.;XMO'.'.,c:,,;cc,:XMMMMMMMMMMMMMMM
# MMWl.,,'''',,;:ccllllloooolllcccccll:.'OMMMMMMMMMMMMMMMMMMMx'.';;,,,,,,,,;,.lWMNo....:c:::c;;OWMMMMMMMMMMMMMMM
# MMWx'',,,,',,,,;;;;;;;;;;;;;;;:::::::' :XMMMMMMMMMMMMMMMMMMK; ';;,,,,,,,,;;.lWMMNkoodoc;;,,,dWMMMMMMMMMMMMMMMM
# MMMK;.'',;;;,,,,,,,;;::cllooolllllllc, .:lodxOKNWMMMMMMMMMMNo..;;,,,,,,,,;;.;KMMMMMMMWXKXKOKWMMMMMMMMMMMMMMMMM
# MMMWO;.'',;;:::cccllllllolllcc;,''......'''''',;:lodxxxddoc;'.;:;,,,,,,,,;ll':KMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
# MMMMMKl'..,,;;:::cccccccc:,'..,cc. ..,,,;;;;;;;;colcc:::;,,;:::;;,,,,,,,,c0Nx,:KMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
# MMMMMMW0o;...'',,,,,,''..',cd0Xk;..,,,,,,,,,,,,;xNWWWWNNKd:;,,,,,,,,,,,,,c0MNd.cXMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMN0dl:;,,,,,;:ldkKNMMNd..',,,,,,,,,,,,,,cONMMMMMWO:,,,,,,,,,,,,,,;oXWXc.xWMMMMMMMMMMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMWWNXXNNWMMMMMMMKc..',,,,,,,,,,,,,,,,:dXMMMMWx;,,,,,,,,,,,,,,,;xNWx.'oOXWMMMMMMMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMM0;..,,,,,,,,,,,,,,,,,,,;kWMMW0c,,,,,,,,,,,,,,,,,c0WO'  .'cxKWMMMMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMW0;'cc;;,,,,,,,,,,,,,,,,;oKMMWWk:,,,,,,,,,,,,,,,,,:kNx. ......:d0WMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMNx,;OWNKOxdlc:::;;::::cldONMMMMWKo;,,,,,,,,,,,,,,,,,:c'.,c:;'.....;dKWMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMWOc'lKWMMMMMWWNXKK00KKKXNWMMMMMMMMWXkocccloddxddolc:;,,'..';oxdl;'... .cONMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMNkc;lONMMMMMMMMMMMNOokXNWWWMMMMMMMMMMMMWNNNNWMMMMWWNNKOo:;,,''',cdxdc;'....;xXMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMNd:lxKWMMMMMMMMMMWXxc,',:ok0KXXXNNNWWWWWWWWWWWWWWWN0llxXWWKko:,,,,.';lxkxdddo:',dXWMMMMMMMMM
# MMMMMMMMMMMMMMMMMM0,;XMMMMMMMMMMWXOl:;loc.  .:ccodkO000KKKKKKKK00Okdl;'.,;lONMN0xc;,,,'.,cdO0000kc',dXMMMMMMMM
# MMMMMMMMMMMMMMMMMM0,:XMMMMMMMNKkolcldxxdc'.,x0kdlc:;;;;;;;;;;;,,,,,;cokXN0l;:xKWWN0kdolodxolok000kc'.,dNMMMMMM
# MMMMMMMMMMMMMMMMMMx'oWWNX0kxdoloxO00Odc'.;xXMMMMMWNK0kxxddooddxkO0XNWMMMMMWKd;,ckNWMMWWWMMWKxllkx:,;:,.;kNMMMM
# MMMMMMMMMMMMMMMMMX:'OMKccl''dO00Oxoc;;:o0WMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMNk:.;xKWMMMMMMMMXx:'.,;;:;..dWMMM
# MMMMMMMMMMMMMMMMWx'lNMO;dNc'x000o';dOKWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWOc.,dKNWMMMMMNkc;..,,'..lKMMMM
# MMMMMMMMMMMMMMMMK;,0MMO;xNc'x000l.dWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWOc.,d0NWMMNd:odo;...:0WMMMMM
# MMMMMMMMMMMMMMMXc'xWMMx:ON:'x000o'xMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMWO:.;dKNWx,;codo'.dNMMMMMMM
# MMMMMMMMMMMMMMKc'dNMMNlcXX:,k000l,OMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMNk;.;xk:':::,.;kNMMMMMMMM
# MMMMMMMMMMMMW0:.lKNNNk;dWO,c0000::KMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMXd'.,';:'.,dXMMMMMMMMMM
# MMMMMMMMMMMMNo.;cllol,:KNc,x0OOd,lNMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMW0:. .':xXMMMMMMMMMMMM
# MMMMMMMMMMMMMXx:'':oc;kWk',cc:;.'kMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMNOod0WMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMNkl:;:kWX:.',,,'.cXMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMNXXWMXd:;,,,,c0MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMWWNXXNWMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM
# MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM


# Last updated 6/5/2025 SW
Sitemap: https://www.splunk.com/en_us/sitemap.xml
Sitemap: https://www.splunk.com/ja_jp/sitemap.xml
Sitemap: https://www.splunk.com/de_de/sitemap.xml
Sitemap: https://www.splunk.com/fr_fr/sitemap.xml
Sitemap: https://www.splunk.com/zh_cn/sitemap.xml
Sitemap: https://www.splunk.com/zh_tw/sitemap.xml
Sitemap: https://www.splunk.com/ko_kr/sitemap.xml
Sitemap: https://www.splunk.com/en_us/blog/sitemap.xml
Sitemap: https://www.splunk.com/ja_jp/blog/sitemap.xml
Sitemap: https://www.splunk.com/de_de/blog/sitemap.xml
Sitemap: https://www.splunk.com/fr_fr/blog/sitemap.xml


User-agent: *
Disallow: /.well-known/assetlinks.json
Disallow: /afc5f206c8ba6de0de90.woff2
Disallow: /api/bin/careers/filters
Disallow: /api/bin/careers/jobs
Disallow: /api/bin/lead/auth/token
Disallow: /api/bin/social/insta
Disallow: /blog/error/404.html
Disallow: /blog/feed/rss.html
Disallow: /blog/feed/rss/
Disallow: /blog/robots.txt
Disallow: /content/dam/images/we_splunk/benefits.js?v1
Disallow: /content/dam/images/we_splunk/new-canvas/de_homepage.js?v1
Disallow: /content/dam/images/we_splunk/new-canvas/fr_homepage.js?v2
Disallow: /content/dam/images/we_splunk/new-canvas/kr_homepage.js?v5
Disallow: /content/dam/images/we_splunk/new-canvas/tw_homepage.js?v4
Disallow: /content/dam/splunk2/en_us/images/ciso-sales-tool-assets/ciso-page-filter.json
Disallow: /content/dam/splunk2/images/icons/favicons/favicon.ico?301=/favicon.ico
Disallow: /content/dam/splunk2/images/logos/splunk-logo.svg?asd
Disallow: /d6bd4261cfad5a82c31c.woff2
Disallow: /en_us/blog/tag/splunk-apps-and-add-ons.html
Disallow: /en_us/careers/search-jobs.html?
Disallow: /etc.clientlibs/splunk/core/components/content/global-nav/header/v2/header/clientlibs/resources/resources/fonts/icomoon/icomoon-v1.ttf
Disallow: /etc.clientlibs/splunk/core/components/content/global-nav/header/v2/header/clientlibs/resources/resources/fonts/icomoon/icomoon-v1.woff
Disallow: /etc.clientlibs/splunk/core/components/content/video/v1/video/clientlibs.lc-d41d8cd98f00b204e9800998ecf8427e-lc.min.css
Disallow: /libs/clientlibs/granite/richtext/resources/images/blank.png
Disallow: /libs/granite/csrf/token.json
Disallow: /my-account/
Disallow: /page/company_news/rss
Disallow: /page/securityportal/rss
Disallow: /saml2/authenticate/okta
Disallow: */blog/fragments/*

Change history

  1. initial snapshot
    • First snapshot of robots.txt archived