NeuralCrawl

Evonik / robots.txt snapshot

← back to evonik.com · fetched 2026-07-23T14:11:06Z (14d ago) · HTTP 200 · 1254 bytes · sha256 4c9bbed724c6d68c · raw

final URL: https://www.evonik.com/robots.txt

1# Start Group: crawler / LLMO
2User-Agent: Bingbot
3User-Agent: GPTBot
4User-Agent: ClaudeBot
5User-Agent: Claude-User
6User-Agent: Claude-SearchBot
7User-Agent: CCBot
8User-Agent: Google-Extended
9User-Agent: Applebot-Extended
10User-Agent: Facebookbot
11User-Agent: Meta-ExternalAgent
12User-Agent: Meta-ExternalFetcher
13User-Agent: diffbot
14User-Agent: PerplexityBot
15User-Agent: TikTokSpider
16User-Agent: Amazonbot
17User-Agent: OAI-SearchBot
18User-Agent: Google-CloudVertexBot
19User-Agent: DuckAssistBot
20User-Agent: GoogleAgent-Mariner
21User-Agent: Gemini-Deep-Research
22User-Agent: Google-NotebookLM
23User-Agent: Google-Agent
24User-Agent: GoogleAgent-URLContext
25User-Agent: Google-Firebase
26User-Agent: MistralAI-User
27User-Agent: Spacecat/1.0
28Disallow: /en/my-evonik/
29Disallow: /en/incident/
30Disallow: /de/incident/
31Disallow: /en/error/
32Allow: /
33# End Group: crawler / LLMO
34# Start Group: known Spam-Bots
35User-agent: MJ12bot
36Disallow: /
37# End Group: known Spam-Bots
38# Start Group: all others
39User-agent: *
40Disallow: /en/my-evonik/
41Disallow: /en/incident/
42Disallow: /de/incident/
43Disallow: /en/error/
44Disallow: /content/dam/
45Disallow: /*j_security_check
46Allow: /
47# End Group: all
48
49Sitemap: https://www.evonik.com/en.sitemap.xml
50Sitemap: https://www.evonik.com/de.sitemap.xml