NeuralCrawl

Edison International / robots.txt snapshot

← back to edison.com · fetched 2026-06-20T01:10:30Z (17h ago) · HTTP 200 · 441 bytes · sha256 118c8ce0f30dff82 · raw

final URL: https://www.edison.com/robots.txt

1User-agent: *
2Allow: /
3Disallow: /*?preview=*
4
5# OpenAI
6User-agent: GPTBot
7Allow: /
8
9User-agent: ChatGPT-User
10Allow: /
11
12User-agent: OAI-SearchBot
13Allow: /
14
15User-agent: OAI-AdsBot
16Allow: /
17
18# Anthropic
19User-agent: ClaudeBot
20Allow: /
21
22# Google AI / Gemini-related usage
23User-agent: Google-Extended
24Allow: /
25
26# Other common AI crawlers
27User-agent: PerplexityBot
28Allow: /
29
30User-agent: CCBot
31Allow: /
32
33Sitemap: https://www.edison.com/sitemap.xml