NeuralCrawl

Qwant / robots.txt snapshot

← back to qwant.com · fetched 2026-06-20T14:56:27Z (4h ago) · HTTP 200 · 2280 bytes · sha256 d768f6d48418d5be · raw

final URL: https://www.qwant.com/robots.txt

1#Robots.txt - www.qwant.com
2
3User-agent: *
4Disallow: /?*q=*
5Disallow: /account
6
7#AI Bots
8User-agent: AI2Bot
9User-agent: AI2Bot-DeepResearchEval
10User-agent: Ai2Bot-Dolma
11User-agent: Amazonbot
12User-agent: AmazonBuyForMe
13User-agent: anthropic-ai
14User-agent: Applebot-Extended
15User-agent: bigsur.ai
16User-agent: Bytespider
17User-agent: CCBot
18User-agent: ChatGLM
19User-agent: ChatGLM-Spider
20User-agent: ChatGPT-Operator
21User-agent: ChatGPT-User
22User-agent: ClaudeBot
23User-agent: Claude-User
24User-agent: Claude-Web
25User-agent: CloudVertexBot
26User-agent: cohere-ai
27User-agent: cohere-training-data-crawler
28User-agent: Cotoyogi
29User-agent: DataForSeoBot
30User-agent: Datenbank Crawler
31User-agent: Devin
32User-agent: Diffbot
33User-agent: DuckAssistBot
34User-agent: FacebookBot
35User-Agent: FriendlyCrawler
36User-agent: gemini-deep-research
37User-agent: GoogleAgent-Mariner
38User-agent: Google-Extended
39User-agent: Google-NotebookLM
40User-agent: GPTBot
41User-agent: ICC-Crawler
42User-agent: ImagesiftBot
43User-agent: imageSpider
44User-agent: img2dataset
45User-agent: Kangaroo Bot
46User-agent: KlaviyoAIBot
47User-agent: laion-huggingface-processor
48User-agent: LCC
49User-agent: LinerBot
50User-agent: Manus-User
51User-agent: meta-externalagent
52User-agent: meta-externalads
53User-agent: meta-externalfetcher
54User-agent: meta-webindexer
55User-agent: MistralAI-User
56User-agent: netEstate
57User-agent: netEstate Imprint Crawler
58User-agent: NovaAct
59User-agent: omgili
60User-agent: OmigiliBot
61User-agent: PanguBot
62User-agent: Perplexity-User
63User-agent: PhindBot
64User-agent: Poggio-Citations
65User-agent: QualifiedBot
66User-agent: SBIntuitionsBot
67User-Agent: SemrushBot-SWA
68User-agent: sider.ai
69User-agent: Spider
70User-agent: TavilyBot
71User-agent: TheKnowledgeAI
72User-agent: TimpiBot
73User-agent: TwinAgent
74User-agent: VelenPublicWebCrawler
75User-agent: webzio-extended
76User-agent: YouBot
77Disallow: /?*q=*
78Disallow: /?drawer=*
79
80#Bots
81User-agent: AhrefsBot
82User-agent: SearchmetricsBot
83User-agent: rogerbot
84User-agent: deepcrawl
85User-Agent: OnCrawl
86User-agent: Screaming Frog SEO Spider
87User-agent: MJ12bot
88User-Agent: Sistrix
89User-agent: barkrowler
90Disallow: /
91
92#Ads Bots
93User-agent: Mediapartners-Google
94User-agent: Google-Display-Ads-Bot
95Disallow:
96
97#Social Bots
98User-agent: Twitterbot
99User-agent: facebookexternalhit
100Disallow: