NeuralCrawl

๐Ÿ‡ฎ๐Ÿ‡ช Ryanair

ryanair.com · European companies · rank #102 · Airlines · live robots.txt ↗

AI crawler access (latest snapshot, 13h ago)

blocked restricted allowed faded = inherited from the * wildcard group

GPTBot
ChatGPT-User
OAI-SearchBot
ClaudeBot
Claude-User
Claude-SearchBot
anthropic-ai
Claude-Web
CCBot
Google-Extended
Applebot-Extended
PerplexityBot
Perplexity-User
Bytespider
Amazonbot
FacebookBot
meta-externalagent
meta-externalfetcher
cohere-ai
AI2Bot
Diffbot
omgili
YouBot
DuckAssistBot
MistralAI-User
PanguBot
Timpibot

Current robots.txt 718 bytes · sha256 daea25f27afe · raw

user-agent: *
allow: /
 
disallow: /*/*/deactive-account
disallow: /*/*/emailChangeConfirm
disallow: /*/*/forgotpassword
disallow: /*/*/reset-password
disallow: /*/*/resetsuccess
disallow: /*/*/reset
disallow: /*/*/lock-account
disallow: /*/*/unlock
disallow: /*/*/verifyemail
disallow: /*/*/reactivate
disallow: /*/*/reactivate-account-succeed
disallow: /*/*/booking
disallow: /*/*/mytrips
disallow: /*/*/paypal
disallow: /*/*/samsonite/
disallow: /*/*/vouchers/
disallow: /*/*/?cmpid=
disallow: /*/*/foh.
disallow: /*/*/rooms/
disallow:/*?promo-code=ABCDE
disallow: /*/*/trip
disallow: /*/*/search/flights
disallow: /redirect
disallow: /*/*/boarding-pass
disallow: /api

sitemap: https://www.ryanair.com/sitemap.xml

Change history

  1. initial snapshot
    • First snapshot of robots.txt archived