🇯🇵 Rakuten

rakuten.com · E-commerce · rank #8 · E-commerce · live robots.txt ↗

AI crawler access (latest snapshot, 3h ago)

⛔blocked restricted ✅allowed faded = inherited from the * wildcard group

✅GPTBot

✅ChatGPT-User

✅OAI-SearchBot

✅ClaudeBot

✅Claude-User

✅Claude-SearchBot

✅anthropic-ai

✅Claude-Web

✅CCBot

✅Google-Extended

✅Applebot-Extended

✅PerplexityBot

✅Perplexity-User

✅Bytespider

✅Amazonbot

✅FacebookBot

✅meta-externalagent

✅meta-externalfetcher

✅cohere-ai

✅AI2Bot

✅Diffbot

✅omgili

✅YouBot

✅DuckAssistBot

✅MistralAI-User

✅PanguBot

✅Timpibot

Current robots.txt 2621 bytes · sha256 8bccbc078da0 · raw

User-agent: *
Disallow: *disallow=1*
Disallow: /.well-known/
Disallow: /*-xfas
Disallow: /*-xfcat
Disallow: /product/
Disallow: /prod-cats/
Disallow: /editUserInfos.do
Disallow: /utilities/
Disallow: /landing_merchant.jsp
Disallow: /2010.htm
Disallow: /compare/
Disallow: /searchproducts.htm
Disallow: /searchproducts_all.htm
Disallow: /searchcomparison.htm
Disallow: /search/special.htm
Disallow: /search/all.htm
Disallow: /search/stores.htm
Disallow: /myus/
Disallow: /account-info.htm
Disallow: /whitelist/
Disallow: /*?referrerid
Disallow: /*&campaign_entity_id
Disallow: /*&sourceName
Disallow: /*&eeid
Disallow: /*&sourceId
Disallow: /*&src
Disallow: /*&utm_source
Disallow: /*&utm_medium
Disallow: /*&utm_campaign
Disallow: /*&AutoLoginID
Disallow: /*&mpl_id
Disallow: /*&referrid
Disallow: /*&referrerdid
Disallow: /*&referrer
Disallow: /unifiedSearch/
Disallow: /refer-a-friend/
Disallow: /frontendEvents.do
Disallow: /button/
Disallow: /giveaway_rules/
Disallow: /wp-json/
Disallow: /feed/merchant-feed.xml
Disallow: /feed/offer-feed.xml
Disallow: /customer/feed/offer-feed.xml
Disallow: /customer/feed/merchant-feed.xml
Disallow: /logonUserServicesStartupAction.do
Disallow: /myaccount/
Disallow: /pending-cash-back.htm
Disallow: /sem/
Disallow: /ebatesCaptcha.do
Disallow: /mobile/
Disallow: /cgi-bin/
Disallow: /blog-preview/
Disallow: /blog/wp-includes/
Disallow: /blog/wp-login.php
Disallow: /blog/wp-admin/*
Disallow: /blog/wp-register.php
Disallow: /blog/search/*
Disallow: /email/
Disallow: /account/
Disallow: /ajax/
Disallow: /auth/getLogonForm.do
Disallow: /zh/index.htm
Disallow: /kr/index.htm
Disallow: /sg/index.htm
Disallow: /sem2015/
Disallow: /ad-scans/
Disallow: /_stores/
Disallow: /hotels/search-results.htm
Disallow: /hotels/verify.htm
Disallow: /hotels/reservation-management.htm
Disallow: /hotels/reservation-payment.htm
Disallow: /hotels/reservation-confirmation.htm
Disallow: /kr/
Disallow: /blog1/
Disallow: *bvstate=
Disallow: *maxp=
Disallow: *minp=
Disallow: *rtg=
Disallow: *po_c=
Disallow: *bo=
Disallow: *queryId=
Disallow: *querySubId=
Disallow: /*.vm
Disallow: /sorting/
Disallow: /m/
Disallow: /shop/search/
Disallow: /shop/*/search/
Disallow: /shop/member/
Disallow: /shop/order/
Disallow: /shop/messages/
Disallow: /shop/wishlist/
Disallow: /shop/mycoupons/
Disallow: /shop/browsing_history/
Disallow: /shop/preference-center-rmail/
Disallow: /amexblue/

User-agent: naverbot
Allow: /kr/
User-agent: yeti
Allow: /kr/
User-agent: grapeshot
Allow: /shop/search/

Sitemap: https://www.rakuten.com/sitemap-index.xml
Sitemap: https://www.rakuten.com/products/product_sitemap.xml

Change history

initial 3h ago snapshot
- First snapshot of robots.txt archived