NeuralCrawl

Itaú Unibanco / robots.txt snapshot

← back to itau.com.br · fetched 2026-10-30T17:56:11Z (7d ago) · HTTP 200 · 3311 bytes · sha256 3f6c3c9fcdafdf7d · raw

final URL: https://www.itau.com.br/robots.txt

1###############################################
2# robots.txt - Permitir indexação e leitura por LLMs e buscadores
3# Última atualização: 05/2026
4###############################################
5
6# Aplicar regras gerais a todos os robôs
7User-agent: *
8Allow: / # Permite acesso geral ao site
9Disallow: /mapa-fst/ # Bloqueia painel internos
10Disallow: /xml
11Disallow: /Xml/
12Disallow: /404/
13Disallow: /*.pdf$
14Disallow: /_arquivosestaticos/
15Disallow: /content/itau/resultado-de-busca-generico
16Disallow: /private/
17Disallow: /auth/
18Disallow: /login/
19Disallow: /minhaconta/
20Disallow: /api/internal/
21Disallow: /admin/
22
23# Exceções — liberar arquivos que são necessários para o funcionamento visual do site
24Allow: /investimentos/
25Allow: /api/public/
26Allow: /api/hub-content/
27Allow: /api/llm-access/
28Allow: /static/images/
29Allow: /static/assets/
30
31
32###############################################
33# Permissões específicas para LLMs (modelos de IA)
34# Esses agentes pertencem a sistemas de IA generativa e buscadores com extensão LLM
35###############################################
36
37# OpenAI / ChatGPT
38User-agent: OAI-SearchBot
39Allow: /
40Allow: /investimentos/
41Disallow: /login/
42Disallow: /private/
43Disallow: /api/internal/
44
45# Google / Gemini (para acesso ampliado de IA)
46User-agent: Google-Extended
47Allow: /
48
49# Anthropic / Claude
50User-agent: ClaudeBot
51Allow: /
52Allow: /investimentos/
53Disallow: /auth/
54Disallow: /api/internal/
55
56# Perplexity AI
57User-agent: PerplexityBot
58Allow: /
59Allow: /investimentos/
60Disallow: /auth/
61Disallow: /api/internal/
62
63# Microsoft Bing / Copilot
64User-agent: BingBot
65Allow: /
66
67# Apple / Applebot e Applebot-Extended (para Siri, Spotlight, Apple Intelligence)
68User-agent: Applebot
69Allow: /
70User-agent: Applebot-Extended
71Allow: /
72
73# Meta (Facebook / Instagram)
74User-agent: facebookexternalhit
75Allow: /
76User-agent: meta-externalagent
77Allow: /
78
79# LinkedIn
80User-agent: LinkedInBot
81Allow: /
82
83# Amazon (Alexa, AWS, Amazon Search)
84User-agent: Amazonbot
85Allow: /
86
87# Common Crawl (fornece muitos conjuntos de dados usados em IA)
88User-agent: CCBot
89Allow: /
90
91
92Crawl-delay: 10
93
94###############################################
95# Sitemap — referência oficial de URLs do site
96###############################################
97
98Sitemap: https://itau.com.br/investimentos/sitemap.xml
99Sitemap: https://itau.com.br/investimentos/sitemap-home.xml
100Sitemap: https://itau.com.br/investimentos/tipos-de-investimentos/sitemap.xml
101Sitemap: https://itau.com.br/investimentos/analises/sitemap.xml
102Sitemap: https://itau.com.br/investimentos/carteiras-recomendadas/sitemap.xml
103Sitemap: https://itau.com.br/investimentos/noticias/sitemap.xml
104Sitemap: https://itau.com.br/investimentos/blog/sitemap.xml
105Sitemap: https://itau.com.br/investimentos/impactos-de-mercado/sitemap.xml
106Sitemap: https://itau.com.br/investimentos/servicos/sitemap.xml
107
108###############################################
109
110# Itau Unibanco
111Sitemap: https://www.itau.com.br/sitemap.xml
112
113###############################################
114# Recomendações adicionais (opcionais)
115# - Adicione no <head> das páginas:
116# <meta name="robots" content="index, follow">
117# <meta name="ai-access" content="allow">
118# - Esses metadados ajudam novos crawlers de IA a identificar permissão explícita.
119###############################################