NeuralCrawl

๐Ÿ‡ง๐Ÿ‡ท Olx

olx.com.br · Top 1000 websites · rank #133 · Web · live robots.txt ↗

AI crawler access (latest snapshot, 1h ago)

blocked restricted allowed faded = inherited from the * wildcard group

GPTBot
ChatGPT-User
OAI-SearchBot
ClaudeBot
Claude-User
Claude-SearchBot
anthropic-ai
Claude-Web
CCBot
Google-Extended
Applebot-Extended
PerplexityBot
Perplexity-User
Bytespider
Amazonbot
FacebookBot
meta-externalagent
meta-externalfetcher
cohere-ai
AI2Bot
Diffbot
omgili
YouBot
DuckAssistBot
MistralAI-User
PanguBot
Timpibot

Current robots.txt 12916 bytes · sha256 4efdd21d579b · raw

User-agent: *
Disallow: /*?*xtmc=*
Disallow: /*xtmc=*
Disallow: /*?*xtcr=*
Disallow: /*xtcr=*
Disallow: /*?*xtnp=*
Disallow: /*xtnp=*
Disallow: /*?*estado-*
Disallow: /q/*
Disallow: /perfil/ofertanacional-com-br-38b5fc95
Disallow: /*?*r_id=*
Disallow: /*?*TestAB_Groups=*
Disallow: /*?*nl_id=*
Disallow: /*?*hjSessionUser_*
Disallow: /*?*hjid=*
Disallow: /*?*ym_uid=*
Disallow: /*?*ym_d=*
Disallow: /*?*ga=*
Disallow: /*?*ga_50C013M2CC=*
Disallow: /*?*QSI_CT=*
Disallow: /*?*uetvid=*
Disallow: /*?*xtvrn=*
Disallow: /*?*ssm_au=*
Disallow: /*?*pbjs_userid_consent_data=*
Disallow: /*?*olx-cookies-accept=*
Disallow: /*?*pin_unauth=*
Disallow: /*?*hjSessionUser_2633764=*
Disallow: /*?*gcl_au=*
Disallow: /*?*loginIdentifier=*
Disallow: /*?*ubvt=*
Disallow: /*?*gads=*
Disallow: /*?*hjSessionUser_1923708=*
Disallow: /*?*l_id=*
Disallow: /*?*blueID=*
Disallow: /*r_id=*
Disallow: /*TestAB_Groups=*
Disallow: /*nl_id=*
Disallow: /*hjSessionUser_*
Disallow: /*hjid=*
Disallow: /*ym_uid=*
Disallow: /*ym_d=*
Disallow: /*ga=*
Disallow: /*ga_50C013M2CC=*
Disallow: /*QSI_CT=*
Disallow: /*uetvid=*
Disallow: /*xtvrn=*
Disallow: /*ssm_au=*
Disallow: /*pbjs_userid_consent_data=*
Disallow: /*olx-cookies-accept=*
Disallow: /*pin_unauth=*
Disallow: /*hjSessionUser_2633764=*
Disallow: /*gcl_au=*
Disallow: /*loginIdentifier=*
Disallow: /*ubvt=*
Disallow: /*gads=*
Disallow: /*hjSessionUser_1923708=*
Disallow: /*l_id=*
Disallow: /*blueID=*
Disallow: /*?*adt=*
Disallow: /*?*apartt=*
Disallow: /*?*apcf=*
Disallow: /*?*apf=*
Disallow: /*?*apt=*
Disallow: /*?*bae=*
Disallow: /*?*bas=*
Disallow: /*?*bbt=*
Disallow: /*?*be=*
Disallow: /*?*bf=*
Disallow: /*?*bhe=*
Disallow: /*?*bhs=*
Disallow: /*?*ble=*
Disallow: /*?*bls=*
Disallow: /*?*bot=*
Disallow: /*?*bs=*
Disallow: /*?*bt=*
Disallow: /*?*bwe=*
Disallow: /*?*bws=*
Disallow: /*?*ca=*
Disallow: /*?*cac=*
Disallow: /*?*cad=*
Disallow: /*?*cas=*
Disallow: /*?*cc=*
Disallow: /*?*cf=*
Disallow: /*?*cft=*
Disallow: /*?*cg=*
Disallow: /*?*coe=*
Disallow: /*?*commf=*
Disallow: /*?*commt=*
Disallow: /*?*cond=*
Disallow: /*?*cos=*
Disallow: /*?*cps=*
Disallow: /*?*cpt=*
Disallow: /*?*ct=*
Disallow: /*?*ctp=*
Disallow: /*?*dbg=*
Disallow: /*?*dlrvw=*
Disallow: /*?*elbh=*
Disallow: /*?*elc=*
Disallow: /*?*elcd=*
Disallow: /*?*et=*
Disallow: /*?*exc=*
Disallow: /*?*extk=*
Disallow: /*?*f=*
Disallow: /*?*fb=*
Disallow: /*?*fnc=*
Disallow: /*?*ft=*
Disallow: /*?*fu=*
Disallow: /*?*gb=*
Disallow: /*?*gend=*
Disallow: /*?*gnv=*
Disallow: /*?*gsp=*
Disallow: /*?*homecf=*
Disallow: /*?*homef=*
Disallow: /*?*homet=*
Disallow: /*?*ipe=*
Disallow: /*?*ips=*
Disallow: /*?*jc=*
Disallow: /*?*landf=*
Disallow: /*?*me=*
Disallow: /*?*mf=*
Disallow: /*?*mit=*
Disallow: /*?*motp=*
Disallow: /*?*ms=*
Disallow: /*?*mt=*
Disallow: /*?*o=*
Disallow: /*?*op=*
Disallow: /*?*opst=*
Disallow: /*?*ot=*
Disallow: /*?*owmn=*
Disallow: /*?*own=*
Disallow: /*?*pct=*
Disallow: /*?*pe=*
Disallow: /*?*pnb=*
Disallow: /*?*pnc=*
Disallow: /*?*pnm=*
Disallow: /*?*ps=*
Disallow: /*?*q=*
Disallow: /*?*re=*
Disallow: /*?*rent_type=*
Disallow: /*?*ret=*
Disallow: /*?*rlt=*
Disallow: /*?*roe=*
Disallow: /*?*rop=*
Disallow: /*?*ros=*
Disallow: /*?*rrf=*
Disallow: /*?*rrt=*
Disallow: /*?*rs=*
Disallow: /*?*sct=*
Disallow: /*?*sd=*
Disallow: /*?*se=*
Disallow: /*?*sf=*
Disallow: /*?*sgt=*
Disallow: /*?*snf=*
Disallow: /*?*snt=*
Disallow: /*?*sp=*
Disallow: /*?*ss=*
Disallow: /*?*sst=*
Disallow: /*?*tf=*
Disallow: /*?*tt=*
Disallow: /*?*vb=*
Disallow: /*?*vgm=*
Disallow: /*?*vgt=*
Disallow: /*?*vm=*
Disallow: /*?*vt=*
Disallow: /*?*w=*
Disallow: /*?*wty=*
Disallow: /*?*zn=*
Disallow: /*?*w=*
Disallow: /*?*vb=*
Disallow: /*?*vm=*
Disallow: /*?*vt=*
Disallow: /*?*be=*
Disallow: /*?*bs=*
Disallow: /*?*ca=*
Disallow: /*?*cg=*
Disallow: /*?*et=*
Disallow: /*?*fu=*
Disallow: /*?*gb=*
Disallow: /*?*zn=*
Disallow: /*?*sd=*
Disallow: /*?*me=*
Disallow: /*?*ms=*
Disallow: /*?*f=*
Disallow: /*?*o=*
Disallow: /*?*pe=*
Disallow: /*?*ps=*
Disallow: /*?*q=*
Disallow: /*?*re=*
Disallow: /*?*rs=*
Disallow: /*?*roe=*
Disallow: /*?*ros=*
Disallow: /*?*sf=*
Disallow: /*?*sp=*
Disallow: /*?*sr=*
Disallow: /*?*ss=*
Disallow: /*?*se=*
Disallow: /*?*ctp=*
Disallow: /*?*cc=*
Disallow: /*?*gend=*
Disallow: /*?*cond=*
Disallow: /*?*jc=*
Disallow: /*?*sct=*
Disallow: /*?*ret=*
Disallow: /*?*sst=*
Disallow: /*?*dbg=*
Disallow: /*?*ot=*
Disallow: /*?*btt=*
Disallow: /*?*ct=*
Disallow: /*?*ft=*
Disallow: /*?*rrt=*
Disallow: /*?*apt=*
Disallow: /*?*adt=*
Disallow: /*?*cpt=*
Disallow: /*?*snt=*
Disallow: /*?*snf=*
Disallow: /*?*rlt=*
Disallow: /*?*rop=*
Disallow: /*?*mit=*
Disallow: /*?*cf=*
Disallow: /*?*sgt=*
Disallow: /*?*pct=*
Disallow: /*?*bae=*
Disallow: /*?*bas=*
Disallow: /*?*commf=*
Disallow: /*?*commt=*
Disallow: /*?*op=*
Disallow: /*?*at=*
Disallow: /*?*st=*
Disallow: /*?*relatedAds=*
Disallow: /*?*pos=*
Disallow: /*?*language=*
Disallow: /*?*campaign=*
Disallow: /*?*fis=*
Disallow: /*?*ic=*
Disallow: /*?*lis=*
Disallow: /*?*local=*
Disallow: /*?*message=*
Disallow: /*?*utm_*=*
Disallow: /*?*target=*
Disallow: /*adt=*
Disallow: /*apartt=*
Disallow: /*apcf=*
Disallow: /*apf=*
Disallow: /*apt=*
Disallow: /*bae=*
Disallow: /*bas=*
Disallow: /*bbt=*
Disallow: /*be=*
Disallow: /*bf=*
Disallow: /*bhe=*
Disallow: /*bhs=*
Disallow: /*ble=*
Disallow: /*bls=*
Disallow: /*bot=*
Disallow: /*bs=*
Disallow: /*bt=*
Disallow: /*bwe=*
Disallow: /*bws=*
Disallow: /*ca=*
Disallow: /*cac=*
Disallow: /*cad=*
Disallow: /*cas=*
Disallow: /*cc=*
Disallow: /*cf=*
Disallow: /*cft=*
Disallow: /*cg=*
Disallow: /*coe=*
Disallow: /*commf=*
Disallow: /*commt=*
Disallow: /*cond=*
Disallow: /*cos=*
Disallow: /*cps=*
Disallow: /*cpt=*
Disallow: /*ct=*
Disallow: /*ctp=*
Disallow: /*dbg=*
Disallow: /*dlrvw=*
Disallow: /*elbh=*
Disallow: /*elc=*
Disallow: /*elcd=*
Disallow: /*et=*
Disallow: /*exc=*
Disallow: /*extk=*
Disallow: /*f=*
Disallow: /*fb=*
Disallow: /*fnc=*
Disallow: /*ft=*
Disallow: /*fu=*
Disallow: /*gb=*
Disallow: /*gend=*
Disallow: /*gnv=*
Disallow: /*gsp=*
Disallow: /*homecf=*
Disallow: /*homef=*
Disallow: /*homet=*
Disallow: /*ipe=*
Disallow: /*ips=*
Disallow: /*jc=*
Disallow: /*landf=*
Disallow: /*me=*
Disallow: /*mf=*
Disallow: /*mit=*
Disallow: /*motp=*
Disallow: /*ms=*
Disallow: /*mt=*
Disallow: /*o=*
Disallow: /*op=*
Disallow: /*opst=*
Disallow: /*ot=*
Disallow: /*owmn=*
Disallow: /*own=*
Disallow: /*pct=*
Disallow: /*pe=*
Disallow: /*pnb=*
Disallow: /*pnc=*
Disallow: /*pnm=*
Disallow: /*ps=*
Disallow: /*q=*
Disallow: /*re=*
Disallow: /*rent_type=*
Disallow: /*ret=*
Disallow: /*rlt=*
Disallow: /*roe=*
Disallow: /*rop=*
Disallow: /*ros=*
Disallow: /*rrf=*
Disallow: /*rrt=*
Disallow: /*rs=*
Disallow: /*sct=*
Disallow: /*sd=*
Disallow: /*se=*
Disallow: /*sf=*
Disallow: /*sgt=*
Disallow: /*snf=*
Disallow: /*snt=*
Disallow: /*sp=*
Disallow: /*ss=*
Disallow: /*sst=*
Disallow: /*tf=*
Disallow: /*tt=*
Disallow: /*vb=*
Disallow: /*vgm=*
Disallow: /*vgt=*
Disallow: /*vm=*
Disallow: /*vt=*
Disallow: /*w=*
Disallow: /*wty=*
Disallow: /*zn=*
Disallow: /*w =*
Disallow: /*vb =*
Disallow: /*vm =*
Disallow: /*vt =*
Disallow: /*be =*
Disallow: /*bs =*
Disallow: /*ca =*
Disallow: /*cg =*
Disallow: /*et =*
Disallow: /*fu =*
Disallow: /*gb =*
Disallow: /*zn =*
Disallow: /*sd =*
Disallow: /*me =*
Disallow: /*ms =*
Disallow: /*f =*
Disallow: /*o =*
Disallow: /*pe =*
Disallow: /*ps =*
Disallow: /*q =*
Disallow: /*re =*
Disallow: /*rs =*
Disallow: /*roe =*
Disallow: /*ros =*
Disallow: /*sf =*
Disallow: /*sp =*
Disallow: /*sr =*
Disallow: /*ss =*
Disallow: /*se =*
Disallow: /*ctp =*
Disallow: /*cc =*
Disallow: /*gend =*
Disallow: /*cond =*
Disallow: /*jc =*
Disallow: /*sct =*
Disallow: /*ret =*
Disallow: /*sst =*
Disallow: /*dbg =*
Disallow: /*ot =*
Disallow: /*btt =*
Disallow: /*ct =*
Disallow: /*ft =*
Disallow: /*rrt =*
Disallow: /*apt =*
Disallow: /*adt =*
Disallow: /*cpt =*
Disallow: /*snt =*
Disallow: /*snf =*
Disallow: /*rlt =*
Disallow: /*rop =*
Disallow: /*mit =*
Disallow: /*cf =*
Disallow: /*sgt =*
Disallow: /*pct =*
Disallow: /*bae =*
Disallow: /*bas =*
Disallow: /*commf =*
Disallow: /*commt =*
Disallow: /*op =*
Disallow: /*at=*
Disallow: /*st=*
Disallow: /*relatedAds=*
Disallow: /*pos=*
Disallow: /*language=*
Disallow: /*campaign=*
Disallow: /*fis=*
Disallow: /*ic=*
Disallow: /*lis=*
Disallow: /*local=*
Disallow: /*message=*
Disallow: /*utm_*
Disallow: /*target=*
Disallow: /ar/
Disallow: /cp/
Disallow: /support/
Disallow: /vf/
Disallow: /aw/
Disallow: /st/
Disallow: /redir/
Disallow: /iredir/
Disallow: /url-para-el-mapa/
Disallow: /galery?
Disallow: /aw/
Disallow: /st/
Disallow: /redir/
Disallow: /iredir/
Disallow: /?*ret=*
Disallow: /*?*rrt=*
Disallow: /*?*st=*
Disallow: /*?*gsp=*
Disallow: /posting/
Disallow: /pg/
Disallow: /lojas/
Disallow: /inserir/
Disallow: /about.php
Disallow: /admin/
Disallow: /alerts.php
Disallow: /bs/
Disallow: /cache/
Disallow: /campaigns/
Disallow: /email/
Disallow: /help.php
Disallow: /js/
Disallow: /lib/
Disallow: /login.php
Disallow: /lostpassword.php
Disallow: /myolx/
Disallow: /nav.php
Disallow: /nf/
Disallow: /post/
Disallow: /posting.php
Disallow: /posting_success.php
Disallow: /register.php
Disallow: /stats/
Disallow: /support/
Disallow: /terms.php
Disallow: /widgets/
Disallow: /aa/
Disallow: /ab/
Disallow: /af/
Disallow: /ar/
Disallow: /as/
Disallow: /ay/
Disallow: /az/
Disallow: /be/
Disallow: /bg/
Disallow: /bh/
Disallow: /bi/
Disallow: /bn/
Disallow: /bo/
Disallow: /br/
Disallow: /ca/
Disallow: /co/
Disallow: /cs/
Disallow: /cy/
Disallow: /da/
Disallow: /de/
Disallow: /dz/
Disallow: /el/
Disallow: /en/
Disallow: /eo/
Disallow: /et/
Disallow: /eu/
Disallow: /fa/
Disallow: /fc/
Disallow: /fi/
Disallow: /fj/
Disallow: /fo/
Disallow: /fr/
Disallow: /ft/
Disallow: /fy/
Disallow: /ga/
Disallow: /gd/
Disallow: /gl/
Disallow: /gn/
Disallow: /gu/
Disallow: /ha/
Disallow: /hi/
Disallow: /hr/
Disallow: /hu/
Disallow: /hy/
Disallow: /ia/
Disallow: /id/
Disallow: /ik/
Disallow: /in/
Disallow: /is/
Disallow: /it/
Disallow: /iu/
Disallow: /iw/
Disallow: /ja/
Disallow: /ji/
Disallow: /ka/
Disallow: /kk/
Disallow: /kl/
Disallow: /km/
Disallow: /kn/
Disallow: /ko/
Disallow: /ks/
Disallow: /ku/
Disallow: /ky/
Disallow: /la/
Disallow: /ln/
Disallow: /lo/
Disallow: /lt/
Disallow: /lv/
Disallow: /mi/
Disallow: /mk/
Disallow: /ml/
Disallow: /mn/
Disallow: /mo/
Disallow: /mr/
Disallow: /my/
Disallow: /na/
Disallow: /ne/
Disallow: /nk/
Disallow: /nl/
Disallow: /no/
Disallow: /oc/
Disallow: /om/
Disallow: /or/
Disallow: /pl/
Disallow: /ps/
Disallow: /pt/
Disallow: /qu/
Disallow: /rm/
Disallow: /ru/
Disallow: /rw/
Disallow: /sa/
Disallow: /sd/
Disallow: /sg/
Disallow: /sh/
Disallow: /si/
Disallow: /sk/
Disallow: /sl/
Disallow: /sm/
Disallow: /sn/
Disallow: /so/
Disallow: /sq/
Disallow: /sr/
Disallow: /ss/
Disallow: /st/
Disallow: /su/
Disallow: /sv/
Disallow: /sw/
Disallow: /ta/
Disallow: /te/
Disallow: /tg/
Disallow: /th/
Disallow: /ti/
Disallow: /tk/
Disallow: /tl/
Disallow: /tn/
Disallow: /tr/
Disallow: /ts/
Disallow: /tt/
Disallow: /tw/
Disallow: /ug/
Disallow: /uk/
Disallow: /ur/
Disallow: /uz/
Disallow: /vi/
Disallow: /vo/
Disallow: /wo/
Disallow: /xh/
Disallow: /yo/
Disallow: /za/
Disallow: /zh/
Disallow: /zu/
Disallow: /zz/
Disallow: /*www.olx.com.br*
Disallow: /reply
Disallow: /list_comments/
Disallow: /choosecountry
Disallow: /feedback
Disallow: /auth/
Disallow: /registration/
Disallow: /search/
Disallow: /busca
Disallow: /api/
sitemap: https://www.olx.com.br/sitemap_index.xml
sitemap: https://www.olx.com.br/ac/sitemap_index.xml
sitemap: https://www.olx.com.br/al/sitemap_index.xml
sitemap: https://www.olx.com.br/ap/sitemap_index.xml
sitemap: https://www.olx.com.br/am/sitemap_index.xml
sitemap: https://www.olx.com.br/ba/sitemap_index.xml
sitemap: https://www.olx.com.br/ce/sitemap_index.xml
sitemap: https://www.olx.com.br/df/sitemap_index.xml
sitemap: https://www.olx.com.br/es/sitemap_index.xml
sitemap: https://www.olx.com.br/go/sitemap_index.xml
sitemap: https://www.olx.com.br/ma/sitemap_index.xml
sitemap: https://www.olx.com.br/mt/sitemap_index.xml
sitemap: https://www.olx.com.br/ms/sitemap_index.xml
sitemap: https://www.olx.com.br/mg/sitemap_index.xml
sitemap: https://www.olx.com.br/pa/sitemap_index.xml
sitemap: https://www.olx.com.br/pb/sitemap_index.xml
sitemap: https://www.olx.com.br/pr/sitemap_index.xml
sitemap: https://www.olx.com.br/pe/sitemap_index.xml
sitemap: https://www.olx.com.br/pi/sitemap_index.xml
sitemap: https://www.olx.com.br/rj/sitemap_index.xml
sitemap: https://www.olx.com.br/rn/sitemap_index.xml
sitemap: https://www.olx.com.br/rs/sitemap_index.xml
sitemap: https://www.olx.com.br/ro/sitemap_index.xml
sitemap: https://www.olx.com.br/rr/sitemap_index.xml
sitemap: https://www.olx.com.br/sc/sitemap_index.xml
sitemap: https://www.olx.com.br/sp/sitemap_index.xml
sitemap: https://www.olx.com.br/se/sitemap_index.xml
sitemap: https://www.olx.com.br/to/sitemap_index.xml
sitemap: https://www.olx.com.br/keywords/sitemap_index.xml

Change history

  1. initial snapshot
    • First snapshot of robots.txt archived