🇺🇸 Expedia Group

expedia.com · Top 1000 websites · rank #885 · Travel and Tourism · live robots.txt ↗

AI crawler access (latest snapshot, 13h ago)

⛔blocked restricted ✅allowed faded = inherited from the * wildcard group

✅GPTBot

✅ChatGPT-User

✅OAI-SearchBot

✅ClaudeBot

✅Claude-User

✅Claude-SearchBot

✅anthropic-ai

✅Claude-Web

✅CCBot

✅Google-Extended

✅Applebot-Extended

✅PerplexityBot

✅Perplexity-User

✅Bytespider

✅Amazonbot

✅FacebookBot

✅meta-externalagent

✅meta-externalfetcher

✅cohere-ai

✅AI2Bot

✅Diffbot

✅omgili

✅YouBot

✅DuckAssistBot

✅MistralAI-User

✅PanguBot

✅Timpibot

Current robots.txt 2069 bytes · sha256 d536d28cbb4c · raw

#
# General bots
#
user-agent: *
#hotel
disallow: /Hotel-Search
disallow: /hotel-search
disallow: /search?
disallow: /*/search?
disallow: /ugc/urs/api/hotelreviews/hotel/HOTELID/
disallow: /LastBooking/ElapsedTime
disallow: /*chkin=*
disallow: /*chkout=*
disallow: /HotelCheckout
disallow: /Checkout
disallow: /checkout
disallow: /HotelBookingConfirmation
disallow: /en-au/
disallow: /en-nz/
#flight
disallow: /Flights-Search
disallow: /Flight-Information
disallow: /Flight-SearchResults
disallow: /Flights-BagFees
disallow: /Details
disallow: /FlightCheckout
disallow: /Confirmation-Flight
#package
disallow: /MultiItemCheckout
disallow: /MultiItemBookingConfirmation
disallow: /flexibleshopping
#car
disallow: /carsearch
disallow: /CarCheckout
#activities
disallow: /things-to-do/search?
disallow: /things-to-do/*/search?
disallow: /*/things-to-do/search
disallow: /things-to-do/reviews-external$
#cruise
disallow: /Cruise-Search
disallow: /Cruise-CabinCategory
disallow: /CruiseCheckout
#other
disallow: /legacyGOTO
disallow: /go/
disallow: /info/
disallow: /trips/
disallow: /Search-Results
disallow: /user/createaccount
disallow: /signup?*
disallow: /login?*
#test-seo
disallow: /api/uisprime/track
disallow: /trvl-px/
disallow: /egcs/v2/collect
disallow: /api/ucs/shortlist/
disallow: /api/v4/typeahead/
disallow: /p/home
disallow: /*/p/home
disallow: /auth/
disallow: /*/auth/
disallow: /*/sso/

#
# Ads
#
user-agent: google-hoteladsverifier
user-agent: Mediapartners-Google
user-agent: AdsBot-Google-Mobile
user-agent: AdsBot-Google
user-agent: AdIdxBot
user-agent: OAI-AdsBot
allow: /
#test-seo
disallow: /api/uisprime/track
disallow: /trvl-px/
disallow: /egcs/v2/collect
disallow: /api/ucs/shortlist/
disallow: /api/v4/typeahead/

#
# SemrushBot
#
user-agent: SemrushBot
disallow: /

#
# AI Search
#
user-agent: OAI-SearchBot
user-agent: ChatGPT-User
user-agent: PerplexityBot
user-agent: Perplexity-User
user-agent: Claude-User
user-agent: Claude-SearchBot
allow: /
disallow: /p/home
disallow: /*/p/home
disallow: /auth/
disallow: /*/auth/
disallow: /*/sso/

Change history

initial 13h ago snapshot
- First snapshot of robots.txt archived