🇧🇷 Petrobras

petrobras.com.br · National indices · rank #79 · Energy · live robots.txt ↗

AI crawler access (latest snapshot, 13h ago)

⛔blocked restricted ✅allowed faded = inherited from the * wildcard group

✅GPTBot

✅ChatGPT-User

✅OAI-SearchBot

✅ClaudeBot

✅Claude-User

✅Claude-SearchBot

✅anthropic-ai

✅Claude-Web

✅CCBot

✅Google-Extended

✅Applebot-Extended

✅PerplexityBot

✅Perplexity-User

✅Bytespider

✅Amazonbot

✅FacebookBot

✅meta-externalagent

✅meta-externalfetcher

✅cohere-ai

✅AI2Bot

✅Diffbot

✅omgili

✅YouBot

✅DuckAssistBot

✅MistralAI-User

✅PanguBot

✅Timpibot

Current robots.txt 922 bytes · sha256 3803c55cfa84 · raw

User-Agent: *
crawl-delay: 10

#busca
Allow: /resultado-da-busca
Disallow: /resultado-da-busca?q=*&*
Disallow: /resultado-da-busca?q=*&delta=*&*
Disallow: /resultado-da-busca?*
Disallow: /*/resultado-da-busca?*

# paginacao
Disallow: /*&delta=&&start=

Disallow: /combo/*
Disallow: /*/c/portal/*
Disallow: /c/portal/*
Disallow: /*/group/*
Disallow: /group/*
Disallow: /o/*
Disallow: /*?p_l_back_url*
Disallow: /s/*
Disallow: /*/web/*
Disallow: /web/*
Disallow: /*/web/guest/*
Disallow: /web/guest/*
Disallow: /wiss/*

# Indexação dos documentos
# Disallow: /documents/* 
# Disallow: */documents/* 
# allow: /documents/d/f3a44542-113e-11ee-be56-0242ac120002/
# allow: /pt/documents/d/f3a44542-113e-11ee-be56-0242ac120002/

# Libera favicon
allow: /o/tema-site-externo-petrobras/images/
allow: */o/tema-site-externo-petrobras/images/

# webstories
Disallow: /web-stories?*

Sitemap: https://petrobras.com.br:443/sitemap.xml

Change history

initial 13h ago snapshot
- First snapshot of robots.txt archived