Expedia Group / robots.txt
← back to expedia.com · change detected 1mo ago (2026-09-28T18:34:49Z) · modified +6 −0
What changed
- Added Disallow: /cl/* for *
- Added Disallow: /cl/* for AdIdxBot
- Added Disallow: /cl/* for AdsBot-Google
- Added Disallow: /cl/* for AdsBot-Google-Mobile
- 🤖 AIAdded Disallow: /cl/* for ChatGPT-User
- 🤖 AIAdded Disallow: /cl/* for Claude-SearchBot
- 🤖 AIAdded Disallow: /cl/* for Claude-User
- Added Disallow: /cl/* for google-hoteladsverifier
- Added Disallow: /cl/* for Mediapartners-Google
- Added Disallow: /cl/* for OAI-AdsBot
- 🤖 AIAdded Disallow: /cl/* for OAI-SearchBot
- 🤖 AIAdded Disallow: /cl/* for Perplexity-User
- 🤖 AIAdded Disallow: /cl/* for PerplexityBot
Diff old (2026-06-20T01:10:31Z) → new (2026-09-28T18:34:49Z)
| @@ -61,6 +61,8 @@ | ||
| 61 | 61 | disallow: /auth/ |
| 62 | 62 | disallow: /*/auth/ |
| 63 | 63 | disallow: /*/sso/ |
| 64 | + # /cl/* endpoints removal test | |
| 65 | + Disallow: /cl/* | |
| 64 | 66 | |
| 65 | 67 | # |
| 66 | 68 | # Ads |
| @@ -78,6 +80,8 @@ | ||
| 78 | 80 | disallow: /egcs/v2/collect |
| 79 | 81 | disallow: /api/ucs/shortlist/ |
| 80 | 82 | disallow: /api/v4/typeahead/ |
| 83 | + # /cl/* endpoints removal test | |
| 84 | + Disallow: /cl/* | |
| 81 | 85 | |
| 82 | 86 | # |
| 83 | 87 | # SemrushBot |
| @@ -100,3 +104,5 @@ | ||
| 100 | 104 | disallow: /auth/ |
| 101 | 105 | disallow: /*/auth/ |
| 102 | 106 | disallow: /*/sso/ |
| 107 | + # /cl/* endpoints removal test | |
| 108 | + Disallow: /cl/* |