🇷🇺 Mail

mail.ru · Top 1000 websites · rank #123 · Web · live robots.txt ↗

AI crawler access (latest snapshot, 1h ago)

⛔blocked restricted ✅allowed faded = inherited from the * wildcard group

⛔GPTBot

⛔ChatGPT-User

⛔OAI-SearchBot

⛔ClaudeBot

⛔Claude-User

⛔Claude-SearchBot

⛔anthropic-ai

⛔Claude-Web

⛔CCBot

⛔Google-Extended

⛔Applebot-Extended

⛔PerplexityBot

⛔Perplexity-User

⛔Bytespider

⛔Amazonbot

⛔FacebookBot

⛔meta-externalagent

⛔meta-externalfetcher

⛔cohere-ai

⛔AI2Bot

⛔Diffbot

⛔omgili

⛔YouBot

⛔DuckAssistBot

⛔MistralAI-User

⛔PanguBot

⛔Timpibot

Current robots.txt 433 bytes · sha256 d3fd6768e203 · raw

User-agent: *
Allow: /*.css
Allow: /*.jpg
Allow: /*.gif
Allow: /*.png
Allow: /v/j/*.js
Allow: /$
Disallow: /

User-agent: Googlebot
Allow: /*.css
Allow: /*.jpg
Allow: /*.gif
Allow: /*.png
Allow: /v/j/*.js
Disallow: *?
Disallow: *search
Disallow: *auth
Disallow: *api

User-agent: Yandex
Allow: /*.css
Allow: /*.jpg
Allow: /*.gif
Allow: /*.png
Allow: /v/j/*.js
Allow: /$
Disallow: /

User-agent: Twitterbot
Disallow: /
Allow: /?logo=

Change history

initial 1h ago snapshot
- First snapshot of robots.txt archived