NeuralCrawl

Qonto / robots.txt

← back to qonto.com · change detected 1mo ago (2026-06-29T17:50:44Z) · modified +2 −490

What changed

Diff old (2026-06-20T01:10:30Z) → new (2026-06-29T17:50:44Z)

@@ -1,4 +1,4 @@
1- # robots.txt qonto.com v1.4.20230921 - Energize your Web Crawlers
1+ # robots.txt qonto.com
22 
33  ## Website Section
44  Sitemap: https://qonto.com/sitemap.xml.gz
@@ -15,9 +15,8 @@
1515  ## IBAN Section
1616  Sitemap: https://qonto.com/sitemap-iban.xml.gz
1717 
18- # Block Assets & Tests
1918  User-agent: *
20- Disallow: /*.pdf$
19+ Disallow: /*.pdf
2120  Disallow: /revamp/*
2221  Disallow: /*/ab-test-sales
2322 
@@ -39,490 +38,3 @@
3938  Disallow: */accountants-partners?name=*
4039  Disallow: */blog/search*
4140  Disallow: */integrations_list?*
42-
43- # Block Spam Bots
44- User-agent: AITCSRobot/1.1
45- User-agent: Alexibot
46- User-agent: Amazonbot
47- User-agent: Aqua_Products
48- User-agent: Arachnophilia
49- User-agent: ASpider/0.09
50- User-agent: asterias
51- User-agent: AURESYS/1.0
52- User-agent: b2w/0.1
53- User-agent: BackDoorBot
54- User-agent: BackDoorBot/1.0
55- User-agent: BackRub/.
56- User-agent: Baiduspider-video
57- User-agent: Big Brother
58- User-agent: Bizbot003
59- User-agent: BizBot04 kirk.overleaf.com
60- User-agent: Black Hole
61- User-agent: Black.Hole
62- User-agent: BlackWidow
63- User-agent: BLEXBot
64- User-agent: BlowFish
65- User-agent: BlowFish/1.0
66- User-agent: Bookmark search tool
67- User-agent: Bot mailto:[email protected]
68- User-agent: BotALot
69- User-agent: BotRightHere
70- User-agent: BSpider/1.0 libwww-perl/0.40
71- User-agent: BuiltBotTough
72- User-agent: Bullseye
73- User-agent: Bullseye/1.0
74- User-agent: BunnySlippers
75- User-agent: CACTVS Chemistry Spider
76- User-agent: Cegbfeieh
77- User-agent: ChangeDetection
78- User-agent: Checkbot/x.xx LWP/5.x
79- User-agent: CheeseBot
80- User-agent: CherryPicker
81- User-agent: CherryPickerElite/1.0
82- User-agent: CherryPickerSE/1.0
83- User-agent: ChinaClaw
84- User-agent: combine/0.0
85- User-agent: conceptbot/0.3
86- User-agent: Copernic
87- User-agent: CopyRightCheck
88- User-agent: cosmos
89- User-agent: Crescent
90- User-agent: Crescent Internet ToolPak HTTP OLE Control v.1.0
91- User-agent: Custo
92- User-agent: CyberPatrol SiteCat Webbot
93- User-agent: CyberSpyder/2.1
94- User-agent: Daumoa
95- User-agent: Deweb/1.01
96- User-agent: DISCo
97- User-agent: DISCo Pump 3.0
98- User-agent: DISCo Pump 3.2
99- User-agent: DISCoFinder
100- User-agent: DittoSpyder
101- User-agent: Download Demon
102- User-agent: Download Demon/3.2.0.8
103- User-agent: Download Demon/3.5.0.11
104- User-agent: dumbot
105- User-agent: eCatch
106- User-agent: eCatch/3.0
107- User-agent: EirGrabber
108- User-agent: EmailCollector
109- User-agent: EmailSiphon
110- User-agent: EmailWolf
111- User-agent: EnigmaBot
112- User-agent: EroCrawler
113- User-agent: es
114- User-agent: explorersearch
115- User-agent: Express WebPictures
116- User-agent: Express WebPictures (www.express-soft.com)
117- User-agent: ExtractorPro
118- User-agent: EyeNetIE
119- User-agent: FairAd Client
120- User-agent: FelixIDE/1.0
121- User-agent: fido/0.9 Harvest/1.4.pl2
122- User-agent: Fish-Search-Robot
123- User-agent: Flaming AttackBot
124- User-agent: FlashGet
125- User-agent: FlashGet WebWasher 3.2
126- User-agent: Foobot
127- User-agent: Freecrawl
128- User-agent: FrontPage
129- User-agent: FrontPage [NC,OR]
130- User-agent: Gaisbot
131- User-agent: gcreep/1.0
132- User-agent: GetRight
133- User-agent: GetRight/2.11
134- User-agent: GetRight/3.1
135- User-agent: GetRight/3.2
136- User-agent: GetRight/3.3
137- User-agent: GetRight/3.3.3
138- User-agent: GetRight/3.3.4
139- User-agent: GetRight/4.0.0
140- User-agent: GetRight/4.1.0
141- User-agent: GetRight/4.1.1
142- User-agent: GetRight/4.1.2
143- User-agent: GetRight/4.2
144- User-agent: GetRight/4.2b (Portuguxeas)
145- User-agent: GetRight/4.2c
146- User-agent: GetRight/4.3
147- User-agent: GetRight/4.5
148- User-agent: GetRight/4.5a
149- User-agent: GetRight/4.5b
150- User-agent: GetRight/4.5b1
151- User-agent: GetRight/4.5b2
152- User-agent: GetRight/4.5b3
153- User-agent: GetRight/4.5b6
154- User-agent: GetRight/4.5b7
155- User-agent: GetRight/4.5c
156- User-agent: GetRight/4.5d
157- User-agent: GetRight/4.5e
158- User-agent: GetRight/5.0beta1
159- User-agent: GetRight/5.0beta2
160- User-agent: GetURL.rexx v1.05
161- User-agent: GetWeb!
162- User-agent: Go!Zilla
163- User-agent: Go!Zilla (www.gozilla.com)
164- User-agent: Go!Zilla 3.3 (www.gozilla.com)
165- User-agent: Go!Zilla 3.5 (www.gozilla.com)
166- User-agent: Go-Ahead-Got-It
167- User-agent: Golem/1.1
168- User-agent: GrabNet
169- User-agent: Grafula
170- User-agent: Gromit/1.0
171- User-agent: grub
172- User-agent: HappyFunBot
173- User-agent: Harvest
174- User-agent: Harvest/1.5
175- User-agent: Hatena Antenna
176- User-agent: Hazel's Ferret Web hopper
177- User-agent: hloader
178- User-agent: HMView
179- User-agent: httplib
180- User-agent: HTTrack
181- User-agent: HTTrack 3.0
182- User-agent: HTTrack [NC,OR]
183- User-agent: Huaweisymantecspider
184- User-agent: humanlinks
185- User-agent: H‰m‰h‰kki/0.2
186- User-agent: Image Stripper
187- User-agent: Image Sucker
188- User-agent: inagist.com url crawler
189- User-agent: IncyWincy/1.0b1
190- User-agent: Indy Library
191- User-agent: Indy Library [NC,OR]
192- User-agent: InfoNaviRobot
193- User-agent: Informant
194- User-agent: INGRID/0.1
195- User-agent: InterGET
196- User-agent: Internet Ninja
197- User-agent: Internet Ninja 4.0
198- User-agent: Internet Ninja 5.0
199- User-agent: Internet Ninja 6.0
200- User-agent: Iron33/1.0.2
201- User-agent: IsraeliSearch/1.0
202- User-agent: ITI Spider
203- User-agent: JennyBot
204- User-agent: JetCar
205- User-agent: JOC Web Spider
206- User-agent: JubiiRobot
207- User-agent: jumpstation
208- User-agent: Katipo/1.0
209- User-agent: Kenjin Spider
210- User-agent: Kenjin.Spider
211- User-agent: Keyword Density/0.9
212- User-agent: Keyword.Density
213- User-agent: KIT-Fireball/2.0 libwww/5.0a
214- User-agent: LabelGrab/1.1
215- User-agent: larbin
216- User-agent: larbin ([email protected])
217- User-agent: larbin [email protected]
218- User-agent: larbin_2.6.2 ([email protected])
219- User-agent: larbin_2.6.2 ([email protected])
220- User-agent: larbin_2.6.2 (listonATccDOTgatechDOTedu)
221- User-agent: larbin_2.6.2 ([email protected])
222- User-agent: larbin_2.6.2 [email protected]
223- User-agent: larbin_2.6.2 [email protected]
224- User-agent: larbin_2.6.2 [email protected]
225- User-agent: larbin_2.6.2 listonATccDOTgatechDOTedu
226- User-agent: larbin_2.6.2 [email protected]
227- User-agent: LeechFTP
228- User-agent: LexiBot
229- User-agent: libWeb/clsHTTP
230- User-agent: LinkextractorPro
231- User-agent: linklooker
232- User-agent: LinkScan/8.1a Unix
233- User-agent: LinkScan/8.1a.Unix
234- User-agent: LinkWalker
235- User-agent: LNSpiderguy
236- User-agent: lwp-trivial
237- User-agent: lwp-trivial/1.34
238- User-agent: Mass Downloader
239- User-agent: Mass Downloader/2.2
240- User-agent: Mata Hari
241- User-agent: Mata.Hari
242- User-agent: MediaFox/x.y
243- User-agent: MerzScope
244- User-agent: METAGOPHER
245- User-agent: Microsoft URL Control
246- User-agent: Microsoft URL Control - 5.01.4511
247- User-agent: Microsoft URL Control - 6.00.8169
248- User-agent: Microsoft.URL
249- User-agent: MIDown tool
250- User-agent: MIIxpc
251- User-agent: MIIxpc/4.2
252- User-agent: Mister PiX
253- User-agent: Mister Pix II 2.01
254- User-agent: Mister Pix II 2.02a
255- User-agent: Mister PiX version.dll
256- User-agent: Mister.PiX
257- User-agent: moget
258- User-agent: moget/2.1
259- User-agent: MOMspider/1.00 libwww-perl/0.40
260- User-agent: Motor/0.2
261- User-agent: MSIECrawler
262- User-agent: naver
263- User-agent: Navroad
264- User-agent: NearSite
265- User-agent: NeoScioCrawler
266- User-agent: Net Vampire
267- User-agent: Net Vampire/3.0
268- User-agent: NetAnts
269- User-agent: NetAnts/1.10
270- User-agent: NetAnts/1.23
271- User-agent: NetAnts/1.24
272- User-agent: NetAnts/1.25
273- User-agent: NetCarta CyberPilot Pro
274- User-agent: NetMechanic
275- User-agent: NetScoop/1.0 libwww/5.0a
276- User-agent: NetSpider
277- User-agent: NetZIP
278- User-agent: NetZip Downloader 1.0 Win32(Nov 12 1998)
279- User-agent: NetZip-Downloader/1.0.62 (Win32; Dec 7 1998)
280- User-agent: NetZippy+(http://www.innerprise.net/usp-spider.asp)
281- User-agent: NHSEWalker/3.0
282- User-agent: NICErsPRO
283- User-agent: Nomad-V2.x
284- User-agent: NPbot
285- User-agent: Nutch
286- User-agent: Occam/1.0
287- User-agent: Octopus
288- User-agent: Offline Explorer
289- User-agent: Offline Explorer/1.2
290- User-agent: Offline Explorer/1.4
291- User-agent: Offline Explorer/1.6
292- User-agent: Offline Explorer/1.7
293- User-agent: Offline Explorer/1.9
294- User-agent: Offline Explorer/2.0
295- User-agent: Offline Explorer/2.1
296- User-agent: Offline Explorer/2.3
297- User-agent: Offline Explorer/2.4
298- User-agent: Offline Explorer/2.5
299- User-agent: Offline Navigator
300- User-agent: Offline.Explorer
301- User-agent: OGspider
302- User-agent: Open Text Site Crawler V1.0
303- User-agent: Openbot
304- User-agent: Openfind
305- User-agent: Openfind data gatherer
306- User-agent: Oracle Ultra Search
307- User-agent: PageGrabber
308- User-agent: Papa Foto
309- User-agent: pavuk
310- User-agent: pcBrowser
311- User-agent: PerMan
312- User-agent: PGP-KA/1.2
313- User-agent: ProPowerBot/2.14
314- User-agent: ProWebWalker
315- User-agent: psbot
316- User-agent: Python-urllib
317- User-agent: QueryN Metasearch
318- User-agent: QueryN.Metasearch
319- User-agent: R6_CommentReader
320- User-agent: R6_FeedFetcher
321- User-agent: Radiation Retriever 1.1
322- User-agent: RealDownload
323- User-agent: RealDownload/4.0.0.40
324- User-agent: RealDownload/4.0.0.41
325- User-agent: RealDownload/4.0.0.42
326- User-agent: ReGet
327- User-agent: RepoMonkey
328- User-agent: RepoMonkey Bait & Tackle/v1.01
329- User-agent: Resume Robot
330- User-agent: RMA
331- User-agent: Roverbot
332- User-agent: SafetyNet Robot 0.1
333- User-agent: searchpreview
334- User-agent: Senrigan/xxxxxx
335- User-agent: SiteSnagger
336- User-agent: SlySearch
337- User-agent: SmartDownload
338- User-agent: SmartDownload/1.2.76 (Win32; Apr 1 1999)
339- User-agent: SmartDownload/1.2.77 (Win32; Aug 17 1999)
340- User-agent: SmartDownload/1.2.77 (Win32; Feb 1 2000)
341- User-agent: SmartDownload/1.2.77 (Win32; Jun 19 2001)
342- User-agent: Snooper/b97_01
343- User-agent: Solbot/1.0 LWP/5.07
344- User-agent: sootle
345- User-agent: SpankBot
346- User-agent: spanner
347- User-agent: Spanner/1.0 (Linux 2.0.27 i586)
348- User-agent: spyder3.microsys.com
349- User-agent: Sqworm/2.9.85-BETA (beta_release; 20011115-775; i686-pc-linux
350- User-agent: SuperBot
351- User-agent: SuperBot/3.0 (Win32)
352- User-agent: SuperBot/3.1 (Win32)
353- User-agent: SuperHTTP
354- User-agent: SuperHTTP/1.0
355- User-agent: Surfbot
356- User-agent: suzuran
357- User-agent: Szukacz/1.4
358- User-agent: tAkeOut
359- User-agent: Teleport
360- User-agent: Teleport Pro
361- User-agent: Teleport Pro/1.29
362- User-agent: Teleport Pro/1.29.1590
363- User-agent: Teleport Pro/1.29.1634
364- User-agent: Teleport Pro/1.29.1718
365- User-agent: Teleport Pro/1.29.1820
366- User-agent: Teleport Pro/1.29.1847
367- User-agent: TeleportPro
368- User-agent: Telesoft
369- User-agent: The Intraformant
370- User-agent: The.Intraformant
371- User-agent: TheNomad
372- User-agent: TightTwatBot
373- User-agent: Titan
374- User-agent: toCrawl/UrlDispatcher
375- User-agent: True_Robot
376- User-agent: True_Robot/1.0
377- User-agent: turingos
378- User-agent: TurnitinBot
379- User-agent: UCSD-Crawler
380- User-agent: UnisterBot
381- User-agent: UnwindFetchor/1.0
382- User-agent: URL Control
383- User-agent: URLSpiderPro
384- User-agent: urlck/1.2.3
385- User-agent: URLy Warning
386- User-agent: URLy.Warning
387- User-agent: Valkyrie/1.0 libwww-perl/0.40
388- User-agent: vBSEO
389- User-agent: VCI
390- User-agent: VCI WebViewer VCI WebViewer Win32
391- User-agent: VoidEYE
392- User-agent: Web Image Collector
393- User-agent: Web Sucker
394- User-agent: Web.Image.Collector
395- User-agent: WebAuto
396- User-agent: WebAuto/3.40 (Win98; I)
397- User-agent: WebBandit
398- User-agent: WebBandit/3.50
399- User-agent: WebCapture 2.0
400- User-agent: WebCopier
401- User-agent: WebCopier v.2.2
402- User-agent: WebCopier v2.5
403- User-agent: WebCopier v2.6
404- User-agent: WebCopier v2.7a
405- User-agent: WebCopier v2.8
406- User-agent: WebCopier v3.0
407- User-agent: WebCopier v3.0.1
408- User-agent: WebCopier v3.2
409- User-agent: WebCopier v3.2a
410- User-agent: WebCopy/
411- User-agent: WebCrawler/3.0 Robot libwww/5.0a
412- User-agent: WebEMailExtrac.*
413- User-agent: WebEnhancer
414- User-agent: WebFerret
415- User-agent: WebFetch
416- User-agent: webfetch/2.1.0
417- User-agent: WebFetcher/0.8,
418- User-agent: WebGo IS
419- User-agent: weblayers/0.0
420- User-agent: WebLeacher
421- User-agent: WebLinker/0.0 libwww-perl/0.1
422- User-agent: WebmasterWorld Extractor
423- User-agent: WebmasterWorld Extractor
424- User-agent: WebmasterWorldForumBot
425- User-agent: WebmasterWorldForumBot
426- User-agent: WebMoose/0.0.0000
427- User-agent: WebReaper
428- User-agent: WebReaper [[email protected]]
429- User-agent: WebReaper [[email protected]]
430- User-agent: WebReaper v9.1 - www.otway.com/webreaper
431- User-agent: WebReaper v9.7 - www.webreaper.net
432- User-agent: WebReaper v9.8 - www.webreaper.net
433- User-agent: WebReaper vWebReaper v7.3 - www,otway.com/webreaper
434- User-agent: [email protected]
435- User-agent: WebSauger
436- User-agent: WebSauger 1.20b
437- User-agent: WebSauger 1.20j
438- User-agent: WebSauger 1.20k
439- User-agent: Website eXtractor
440- User-agent: Website Quester
441- User-agent: Website Quester - www.asona.org
442- User-agent: Website Quester - www.esalesbiz.com/extra/
443- User-agent: Website.Quester
444- User-agent: Webster Pro
445- User-agent: Webster.Pro
446- User-agent: WebStripper
447- User-agent: WebStripper/2.03
448- User-agent: WebStripper/2.10
449- User-agent: WebStripper/2.12
450- User-agent: WebStripper/2.13
451- User-agent: WebStripper/2.15
452- User-agent: WebStripper/2.16
453- User-agent: WebStripper/2.19
454- User-agent: WebVac
455- User-agent: webvac/1.0
456- User-agent: webwalk
457- User-agent: WebWalker
458- User-agent: WebWalker/1.10
459- User-agent: WebWatch
460- User-agent: WebWhacker
461- User-agent: WebZIP
462- User-agent: WebZIP/2.75 (http://www.spidersoft.com)
463- User-agent: WebZIP/3.65 (http://www.spidersoft.com)
464- User-agent: WebZIP/3.80 (http://www.spidersoft.com)
465- User-agent: WebZip/4.0
466- User-agent: WebZIP/4.0 (http://www.spidersoft.com)
467- User-agent: WebZIP/4.1 (http://www.spidersoft.com)
468- User-agent: WebZIP/4.21
469- User-agent: WebZIP/4.21 (http://www.spidersoft.com)
470- User-agent: WebZIP/5.0
471- User-agent: WebZIP/5.0 (http://www.spidersoft.com)
472- User-agent: WebZIP/5.0 PR1 (http://www.spidersoft.com)
473- User-agent: Wget
474- User-agent: wget
475- User-agent: Wget/1.4.0
476- User-agent: Wget/1.5.2
477- User-agent: Wget/1.5.3
478- User-agent: Wget/1.6
479- User-agent: Wget/1.7
480- User-agent: Wget/1.8
481- User-agent: Wget/1.8.1
482- User-agent: Wget/1.8.1+cvs
483- User-agent: Wget/1.8.2
484- User-agent: Wget/1.9-beta
485- User-agent: WhoWhere Robot
486- User-agent: Widow
487- User-agent: wired-digital-newsbot/1.5
488- User-agent: WWW Collector
489- User-agent: WWW-Collector-E
490- User-agent: www.freeloader.com.
491- User-agent: WWWOFFLE
492- User-agent: WWWWanderer v3.0
493- User-agent: Xaldon WebSpider
494- User-agent: Xaldon WebSpider 2.5.b3
495- User-agent: Xaldon_WebSpider
496- User-agent: Xenu's
497- User-agent: Xenu's Link Sleuth 1.1c
498- User-agent: XGET/0.7
499- User-agent: Yasaklibot
500- User-agent: yes
501- User-agent: YesupBot
502- User-agent: Yeti
503- User-agent: Zeus
504- User-agent: Zeus 11389 Webster Pro V2.9 Win32
505- User-agent: Zeus 11652 Webster Pro V2.9 Win32
506- User-agent: Zeus 18018 Webster Pro V2.9 Win32
507- User-agent: Zeus 26378 Webster Pro V2.9 Win32
508- User-agent: Zeus 30747 Webster Pro V2.9 Win32
509- User-agent: Zeus 32297 Webster Pro V2.9 Win32
510- User-agent: Zeus 39206 Webster Pro V2.9 Win32
511- User-agent: Zeus 41641 Webster Pro V2.9 Win32
512- User-agent: Zeus 44238 Webster Pro V2.9 Win32
513- User-agent: Zeus 51070 Webster Pro V2.9 Win32
514- User-agent: Zeus 51674 Webster Pro V2.9 Win32
515- User-agent: Zeus 51837 Webster Pro V2.9 Win32
516- User-agent: Zeus 63567 Webster Pro V2.9 Win32
517- User-agent: Zeus 6694 Webster Pro V2.9 Win32
518- User-agent: Zeus 82016 Webster Pro V2.9 Win32
519- User-agent: Zeus 82900 Webster Pro V2.9 Win32
520- User-agent: Zeus 84842 Webster Pro V2.9 Win32
521- User-agent: Zeus 90872 Webster Pro V2.9 Win32
522- User-agent: Zeus 94934 Webster Pro V2.9 Win32
523- User-agent: Zeus 95245 Webster Pro V2.9 Win32
524- User-agent: Zeus 95351 Webster Pro V2.9 Win32
525- User-agent: Zeus 97371 Webster Pro V2.9 Win32
526- User-agent: Zeus Link Scout
527- User-agent: ZyBorg
528- Disallow: /