NeuralCrawl

Qwant / robots.txt snapshot

← back to qwant.com · fetched 2026-08-25T19:41:43Z (24d ago) · HTTP 200 · 2376 bytes · sha256 66d05073c57839eb · raw

final URL: https://www.qwant.com/robots.txt

1#Robots.txt - www.qwant.com
2
3User-agent: *
4Disallow: /?*q=*
5Disallow: /?*drawer=*
6Disallow: /?*bottomMenu=*
7Disallow: /?*home=*
8
9#AI Bots
10User-agent: AI2Bot
11User-agent: AI2Bot-DeepResearchEval
12User-agent: Ai2Bot-Dolma
13User-agent: Amazonbot
14User-agent: AmazonBuyForMe
15User-agent: anthropic-ai
16User-agent: Applebot-Extended
17User-agent: bigsur.ai
18User-agent: Bytespider
19User-agent: CCBot
20User-agent: ChatGLM
21User-agent: ChatGLM-Spider
22User-agent: ChatGPT-Operator
23User-agent: ChatGPT-User
24User-agent: ClaudeBot
25User-agent: Claude-User
26User-agent: Claude-Web
27User-agent: CloudVertexBot
28User-agent: cohere-ai
29User-agent: cohere-training-data-crawler
30User-agent: Cotoyogi
31User-agent: DataForSeoBot
32User-agent: Datenbank Crawler
33User-agent: Devin
34User-agent: Diffbot
35User-agent: DuckAssistBot
36User-agent: FacebookBot
37User-Agent: FriendlyCrawler
38User-agent: gemini-deep-research
39User-agent: GoogleAgent-Mariner
40User-agent: Google-Extended
41User-agent: Google-NotebookLM
42User-agent: GPTBot
43User-agent: ICC-Crawler
44User-agent: ImagesiftBot
45User-agent: imageSpider
46User-agent: img2dataset
47User-agent: Kangaroo Bot
48User-agent: KlaviyoAIBot
49User-agent: laion-huggingface-processor
50User-agent: LCC
51User-agent: LinerBot
52User-agent: Manus-User
53User-agent: meta-externalagent
54User-agent: meta-externalads
55User-agent: meta-externalfetcher
56User-agent: meta-webindexer
57User-agent: MistralAI-User
58User-agent: netEstate
59User-agent: netEstate Imprint Crawler
60User-agent: NovaAct
61User-agent: omgili
62User-agent: OmigiliBot
63User-agent: PanguBot
64User-agent: Perplexity-User
65User-agent: PhindBot
66User-agent: Poggio-Citations
67User-agent: QualifiedBot
68User-agent: SBIntuitionsBot
69User-Agent: SemrushBot-SWA
70User-agent: sider.ai
71User-agent: Spider
72User-agent: TavilyBot
73User-agent: TheKnowledgeAI
74User-agent: TimpiBot
75User-agent: TwinAgent
76User-agent: VelenPublicWebCrawler
77User-agent: webzio-extended
78User-agent: YouBot
79Disallow: /?*q=*
80Disallow: /?*drawer=*
81Disallow: /?*bottomMenu=*
82Disallow: /?*home=*
83
84#Bots
85User-agent: AhrefsBot
86User-agent: SearchmetricsBot
87User-agent: rogerbot
88User-agent: deepcrawl
89User-Agent: OnCrawl
90User-agent: Screaming Frog SEO Spider
91User-agent: MJ12bot
92User-Agent: Sistrix
93User-agent: barkrowler
94Disallow: /
95
96#Ads Bots
97User-agent: Mediapartners-Google
98User-agent: Google-Display-Ads-Bot
99Disallow:
100
101#Social Bots
102User-agent: Twitterbot
103User-agent: facebookexternalhit
104Disallow: