🇯🇵 SoftBank Group

softbank.jp · Top 1000 websites · rank #842 · Telecommunications · live robots.txt ↗

AI crawler access (latest snapshot, 13h ago)

⛔blocked restricted ✅allowed faded = inherited from the * wildcard group

✅GPTBot

✅ChatGPT-User

✅OAI-SearchBot

✅ClaudeBot

✅Claude-User

✅Claude-SearchBot

✅anthropic-ai

✅Claude-Web

✅CCBot

✅Google-Extended

✅Applebot-Extended

✅PerplexityBot

✅Perplexity-User

✅Bytespider

✅Amazonbot

✅FacebookBot

✅meta-externalagent

✅meta-externalfetcher

✅cohere-ai

✅AI2Bot

✅Diffbot

✅omgili

✅YouBot

✅DuckAssistBot

✅MistralAI-User

✅PanguBot

✅Timpibot

Current robots.txt 833 bytes · sha256 bcb3ec81739e · raw

User-agent: *
Disallow:/mobile/campaigns/list/carinsurance-01/-page_02/
Disallow:/*html_include*/
Disallow:/*sub_navi*/
Disallow:/*html_parts*/
Disallow:/online-shop/apply-event/
Disallow:/online-shop/apply/
Disallow:/online-shop/pre-online/
Disallow:/online-shop/pre-shop/
Disallow:/online-shop/pre-shop-t/
Disallow:/online-shop/special/itunes-code/first-campaign/
Disallow:/online-shop/special/promo/
Disallow:/energy/set/common/pdf/terms/*.pdf$
Disallow:/internet/set/data/terms/pdf/*.pdf$
Disallow:/_Metadata/
Disallow:/corp/_Metadata/
Disallow:/mobile/_Metadata/
Disallow:/internet/_Metadata/
Disallow:/biz/_Metadata/
Disallow:/energy/_Metadata/
Disallow:/mysoftbank/_Metadata/
Disallow:/online-shop/_Metadata/
Allow:/online-shop/special/promo/outlet/
Sitemap: https://www.softbank.jp/-/media/sb/sjp/SitemapXML/sitemapindex.xml

Change history

initial 13h ago snapshot
- First snapshot of robots.txt archived