NeuralCrawl

The Telegraph / robots.txt

← back to telegraph.co.uk · change detected 1d ago (2026-11-05T19:53:48Z) · modified +68 −1

What changed

Diff old (2026-11-03T13:00:12Z) → new (2026-11-05T19:53:48Z)

@@ -113,7 +113,74 @@
113113  Allow: /travel/luxury-stays/
114114  Allow: /business/uk-business-growth/
115115  Allow: /ads.txt
116-
116+ Allow: /bah-avis/usa-sports-road-trip-map
117+ Allow: /betting/
118+ Allow: /british-airways-holidays/cyprus-hotel-map
119+ Allow: /business/ai-era-job-skills/
120+ Allow: /business/ai-workflow-transformation/
121+ Allow: /business/future-hr-technology/
122+ Allow: /business/golf-dp-world-tour/
123+ Allow: /business/search-marketing/
124+ Allow: /business/waste-management/
125+ Allow: /deals/
126+ Allow: /environment/woodland-action/
127+ Allow: /fashion/joseph-ribkoff-autumn-collection/
128+ Allow: /financial-services/
129+ Allow: /food-and-drink/m-and-s-seasonal-favourites/
130+ Allow: /food-and-drink/m-and-s-seasonal-favourites/organic-range/
131+ Allow: /food-and-drink/maldon-salt-perfect-pyramid-challenge/
132+ Allow: /food-and-drink/spanish-dining-uk/
133+ Allow: /football/global-football-industry-awards/
134+ Allow: /health-fitness/how-to-choose-private-hospital-hca-healthcare/
135+ Allow: /interiors/next-home-autumn-interiors-trends/
136+ Allow: /junior-golf/
137+ Allow: /lifestyle-solutions/
138+ Allow: /lifestyle/apl-media/
139+ Allow: /branded-content/
140+ Allow: /money/investing/fc-investment-trust/long-term-investing/
141+ Allow: /branded-content/
142+ Allow: /rugby-union/grassroots-rugby-heroes/
143+ Allow: /rugby-union/levelling-the-playing-field/
144+ Allow: /sailgp/inside-f50-catamaran
145+ Allow: /seabourn/explore-the-mediterranean
146+ Allow: /sport/sailgp-championship/
147+ Allow: /travel/active-family-holidays/
148+ Allow: /travel/away-holidays/
149+ Allow: /travel/barbados-holiday-guide/
150+ Allow: /travel/british-airways-american-express/
151+ Allow: /travel/canadian-affair-holidays/rocky-mountaineer-train-route/
152+ Allow: /travel/croatia-holidays/
153+ Allow: /travel/cruises/holland-america-line-luxury-holidays/
154+ Allow: /travel/cruises/seabourn-luxury-cruises/
155+ Allow: /travel/discover-america/
156+ Allow: /travel/exclusiveoffers/
157+ Allow: /branded-content/
158+ Allow: /travel/florida-guide/
159+ Allow: /travel/fly-drive-holidays/
160+ Allow: /travel/grenada-guide/
161+ Allow: /travel/icons-of-the-sky/
162+ Allow: /travel/indian-ocean-escapes/
163+ Allow: /travel/indulge-in-ireland/
164+ Allow: /travel/jamaica-holiday-guide/
165+ Allow: /travel/japan-ninja-heritage/
166+ Allow: /travel/luxury-all-inclusive-resorts/affordable-caribbean-hotels/
167+ Allow: /travel/luxury-holidays/
168+ Allow: /travel/luxury-hotel-getaways/
169+ Allow: /travel/luxury-island-escapes/
170+ Allow: /travel/maldives-holiday-guide/
171+ Allow: /travel/ski-resorts-for-everyone/
172+ Allow: /travel/spark/best-of-switzerland/
173+ Allow: /travel/tailor-made-tours/
174+ Allow: /travel/tenerife-holidays/
175+ Allow: /travel/tips-for-visiting-spain/
176+ Allow: /travel/travel-counsellors/book-a-cruise-with-travel-counsellors/
177+ Allow: /travel/travel-solutions/
178+ Allow: /travel/visit-cayman-islands/
179+ Allow: /travel/why-choose-malta/
180+ Allow: /trusted-partners/
181+ Allow: /brand-usa
182+ Allow: /travel/visit-aarhus/
183+ Allow: /bespoke-content/
117184 
118185  # Sitemap URLs
119186  Sitemap: https://www.telegraph.co.uk/sitemap.xml