🇯🇵 Rakuten

rakuten.co.jp · Top 1000 websites · rank #131 · Web · live robots.txt ↗

AI crawler access (latest snapshot, 2h ago)

⛔blocked restricted ✅allowed faded = inherited from the * wildcard group

✅GPTBot

✅ChatGPT-User

✅OAI-SearchBot

✅ClaudeBot

✅Claude-User

✅Claude-SearchBot

✅anthropic-ai

✅Claude-Web

✅CCBot

✅Google-Extended

✅Applebot-Extended

✅PerplexityBot

✅Perplexity-User

✅Bytespider

✅Amazonbot

✅FacebookBot

✅meta-externalagent

✅meta-externalfetcher

✅cohere-ai

✅AI2Bot

✅Diffbot

✅omgili

✅YouBot

✅DuckAssistBot

✅MistralAI-User

✅PanguBot

✅Timpibot

Current robots.txt 490 bytes · sha256 10c0ca813e13 · raw

User-Agent: *
Disallow: /com/
Disallow: /images/
Disallow: /backup/
Disallow: /cgi-bin/
Disallow: /shops/
Disallow: /shop/ascosing/
Disallow: /shop/egs-g/
Disallow: /shop/tripleo-shop/
Disallow: /shop/fooody/
Disallow: /shop/kira-con02/
Disallow: /*/lightbox_*.html
Disallow: /aboutus/am/maint/
Allow: /com/img/home/logo/touch_google.png
Allow: /com/assets/domain-resources/favicon.ico

User-Agent: AdsBot-Google
Disallow: /com/

User-agent: Googlebot
Disallow: /*?lang=
Disallow: /*&lang=

Change history

initial 2h ago snapshot
- First snapshot of robots.txt archived