NeuralCrawl

Renault Group / robots.txt snapshot

← back to renaultgroup.com · fetched 2026-07-20T19:11:56Z (17d ago) · HTTP 200 · 4150 bytes · sha256 6a04dc0511974c97 · raw

final URL: https://www.renaultgroup.com/robots.txt

1User-Agent: *
2Allow: /
3Disallow: /_next/
4Disallow: /api/
5Disallow: /static/
6Disallow: /wp-content/uploads/
7Disallow: /search/
8Disallow: /de/
9Disallow: /es/
10Disallow: /it/
11Disallow: /pt/
12Disallow: /br/
13Disallow: /ro/
14Disallow: /nl/
15
16# OpenAI — GPTBot (model training) — FR + EN only
17User-agent: GPTBot
18Allow: /
19Disallow: /de/
20Disallow: /es/
21Disallow: /it/
22Disallow: /pt/
23Disallow: /br/
24Disallow: /ro/
25Disallow: /nl/
26
27# OpenAI — OAI-SearchBot (ChatGPT Search real-time results) — FR + EN only
28User-agent: OAI-SearchBot
29Allow: /
30Disallow: /de/
31Disallow: /es/
32Disallow: /it/
33Disallow: /pt/
34Disallow: /br/
35Disallow: /ro/
36Disallow: /nl/
37
38# OpenAI — ChatGPT-User (user-triggered in ChatGPT) — FR + EN only
39User-agent: ChatGPT-User
40Allow: /
41Disallow: /de/
42Disallow: /es/
43Disallow: /it/
44Disallow: /pt/
45Disallow: /br/
46Disallow: /ro/
47Disallow: /nl/
48
49# Anthropic — ClaudeBot (model training) — FR + EN only
50User-agent: ClaudeBot
51Allow: /
52Disallow: /de/
53Disallow: /es/
54Disallow: /it/
55Disallow: /pt/
56Disallow: /br/
57Disallow: /ro/
58Disallow: /nl/
59
60# Anthropic — Claude-Web — FR + EN only
61User-agent: Claude-Web
62Allow: /
63Disallow: /de/
64Disallow: /es/
65Disallow: /it/
66Disallow: /pt/
67Disallow: /br/
68Disallow: /ro/
69Disallow: /nl/
70
71# Anthropic — anthropic-ai — FR + EN only
72User-agent: anthropic-ai
73Allow: /
74Disallow: /de/
75Disallow: /es/
76Disallow: /it/
77Disallow: /pt/
78Disallow: /br/
79Disallow: /ro/
80Disallow: /nl/
81
82# Google AI — FR + EN only
83User-agent: Google-Extended
84Allow: /
85Disallow: /de/
86Disallow: /es/
87Disallow: /it/
88Disallow: /pt/
89Disallow: /br/
90Disallow: /ro/
91Disallow: /nl/
92
93# Perplexity — FR + EN only
94User-agent: PerplexityBot
95Allow: /
96Disallow: /de/
97Disallow: /es/
98Disallow: /it/
99Disallow: /pt/
100Disallow: /br/
101Disallow: /ro/
102Disallow: /nl/
103
104# Apple — FR + EN only
105User-agent: Applebot-Extended
106Allow: /
107Disallow: /de/
108Disallow: /es/
109Disallow: /it/
110Disallow: /pt/
111Disallow: /br/
112Disallow: /ro/
113Disallow: /nl/
114
115# xAI — Grok — FR + EN only
116User-agent: Grok
117Allow: /
118Disallow: /de/
119Disallow: /es/
120Disallow: /it/
121Disallow: /pt/
122Disallow: /br/
123Disallow: /ro/
124Disallow: /nl/
125
126# Meta — FR + EN only
127User-agent: FacebookBot
128Allow: /
129Disallow: /de/
130Disallow: /es/
131Disallow: /it/
132Disallow: /pt/
133Disallow: /br/
134Disallow: /ro/
135Disallow: /nl/
136
137# Microsoft Copilot — FR + EN only
138User-agent: Bingbot
139Allow: /
140Disallow: /de/
141Disallow: /es/
142Disallow: /it/
143Disallow: /pt/
144Disallow: /br/
145Disallow: /ro/
146Disallow: /nl/
147
148# Amazon — FR + EN only
149User-agent: Amazonbot
150Allow: /
151Disallow: /de/
152Disallow: /es/
153Disallow: /it/
154Disallow: /pt/
155Disallow: /br/
156Disallow: /ro/
157Disallow: /nl/
158
159# Common Crawl (open-source models) — FR + EN only
160User-agent: CCBot
161Allow: /
162Disallow: /de/
163Disallow: /es/
164Disallow: /it/
165Disallow: /pt/
166Disallow: /br/
167Disallow: /ro/
168Disallow: /nl/
169
170# ByteDance / TikTok (Doubao and other LLMs) — FR + EN only
171User-agent: Bytespider
172Allow: /
173Disallow: /de/
174Disallow: /es/
175Disallow: /it/
176Disallow: /pt/
177Disallow: /br/
178Disallow: /ro/
179Disallow: /nl/
180
181# OpenAI — GPTBot (model training)
182User-agent: GPTBot
183Allow: /de/
184Allow: /es/
185Allow: /it/
186Allow: /pt/
187Allow: /br/
188Allow: /ro/
189Disallow: /
190
191# OpenAI — OAI-SearchBot (ChatGPT Search real-time results)
192User-agent: OAI-SearchBot
193Allow: /
194
195# OpenAI — ChatGPT-User (user-triggered in ChatGPT)
196User-agent: ChatGPT-User
197Allow: /
198
199# Anthropic — ClaudeBot (model training)
200User-agent: ClaudeBot
201Allow: /de/
202Allow: /es/
203Allow: /it/
204Allow: /pt/
205Allow: /br/
206Allow: /ro/
207Disallow: /
208
209# Anthropic — Claude-Web
210User-agent: Claude-Web
211Allow: /
212
213# Anthropic — anthropic-ai
214User-agent: anthropic-ai
215Allow: /
216
217# Google AI
218User-agent: Google-Extended
219Allow: /
220
221# Perplexity
222User-agent: PerplexityBot
223Allow: /
224
225# Apple
226User-agent: Applebot-Extended
227Allow: /
228
229# xAI — Grok
230User-agent: Grok
231Allow: /
232
233# Meta
234User-agent: FacebookBot
235Allow: /
236
237# Microsoft Copilot
238User-agent: Bingbot
239Allow: /
240
241# Amazon
242User-agent: Amazonbot
243Allow: /
244
245# Common Crawl (open-source models)
246User-agent: CCBot
247Allow: /
248
249# ByteDance / TikTok (Doubao and other LLMs)
250User-agent: Bytespider
251Allow: /
252
253Sitemap: https://www.renaultgroup.com/sitemap.xml
254Sitemap: https://www.renaultgroup.com/sitemap2-index.xml
255Sitemap: https://www.renaultgroup.com/sitemap-en-index.xml