NeuralCrawl

Uol / robots.txt snapshot

← back to uol.com.br · fetched 2026-08-06T19:27:21Z (3mo ago) · HTTP 200 · 2576 bytes · sha256 737b1ef033ac622f · raw

final URL: https://www.uol.com.br/robots.txt

1# robots.txt
2#
3User-agent: *
4Sitemap: https://www.uol.com.br/carros/sitemap/v2/news-01.xml
5Sitemap: https://www.uol.com.br/ecoa/sitemap/news-01.xml
6Sitemap: https://www.uol.com.br/esporte/sitemap/v2/news-01.xml
7Sitemap: https://www.uol.com.br/nossa/sitemap/news-01.xml
8Sitemap: https://www.uol.com.br/splash/sitemap/news-01.xml
9Sitemap: https://www.uol.com.br/tilt/sitemap/news-01.xml
10Sitemap: https://www.uol.com.br/universa/sitemap/v2/news-01.xml
11Sitemap: https://www.uol.com.br/vivabem/sitemap/v2/news-01.xml
12Sitemap: https://c.jsuol.com.br/assets/jupiter-news/?resource-id=sitemap&source=toca/index.xml
13Allow: /
14Disallow: /carros/dev/
15Disallow: /*.htm?uol_app
16Disallow: /*.htm?*&uol_app
17Disallow: /*.ghtm?uol_app
18Disallow: /*.ghtm?*&uol_app
19User-agent: Google-Extended
20Disallow: /
21Allow: /.ghtm$
22User-agent: AI2Bot
23Disallow: /
24User-agent: Amazonbot
25Disallow: /
26User-agent: Anthropic-ai
27Disallow: /
28User-agent: Applebot
29Disallow: /
30User-agent: Applebot-Extended
31Disallow: /
32User-agent: AwarioRssBot
33Disallow: /
34User-agent: AwarioSmartBot
35Disallow: /
36User-agent: Bytespider
37Disallow: /
38User-agent: CCBot
39Disallow: /
40User-agent: Claude-SearchBot
41Disallow: /
42User-agent: Claude-User
43Disallow: /
44User-agent: Claude-Web
45Disallow: /
46User-agent: ClaudeBot
47Disallow: /
48User-agent: Cohere-ai
49Disallow: /
50User-agent: cohere-training-data-crawler
51Disallow: /
52User-agent: Cotoyogi
53Disallow: /
54User-agent: DataForSeoBot
55Disallow: /
56User-agent: Datenbank Crawler
57Disallow: /
58User-agent: DeepSeek
59Disallow: /
60User-agent: Devin
61Disallow: /
62User-agent: Diffbot
63Disallow: /
64User-agent: DuckAssistBot
65Disallow: /
66User-agent: FacebookBot
67Disallow: /
68User-agent: Factset_spyderbot
69Disallow: /
70User-agent: Gemini-Deep-Research
71Disallow: /
72User-agent: GoogleAgent-Mariner
73Disallow: /
74User-agent: ICC-Crawler
75Disallow: /
76User-agent: ImagesiftBot
77Disallow: /
78User-agent: Kangaroo Bot
79Disallow: /
80User-agent: Meta-ExternalAgent
81Disallow: /
82User-agent: Meta-ExternalFetcher
83Disallow: /
84User-agent: MistralAI-User
85Disallow: /
86User-agent: netEstate Imprint Crawler
87Disallow: /
88User-agent: NovaAct
89Disallow: /
90User-agent: omgili
91Disallow: /
92User-agent: Omgilibot
93Disallow: /
94User-agent: Operator
95Disallow: /
96User-agent: PanguBot
97Disallow: /
98User-agent: Peer39_crawler/1.0
99Disallow: /
100User-agent: Perplexity-User
101Disallow: /
102User-agent: PerplexityBot
103Disallow: /
104User-agent: PetalBot
105Disallow: /
106User-agent: Qwen
107Disallow: /
108User-agent: SemrushBot-OCOB
109Disallow: /
110User-agent: Timpibot
111Disallow: /
112User-agent: Twitterbot
113Disallow: /
114User-agent: Webzio-Extended
115Disallow: /
116User-agent: YouBot
117Disallow: /