NeuralCrawl

πŸ‡ΊπŸ‡Έ Fox News

foxnews.com · Publishers · rank #10 · Broadcast News · live robots.txt ↗

AI crawler access (latest snapshot, 13h ago)

blocked restricted allowed faded = inherited from the * wildcard group

GPTBot
ChatGPT-User
OAI-SearchBot
ClaudeBot
Claude-User
Claude-SearchBot
anthropic-ai
Claude-Web
CCBot
Google-Extended
Applebot-Extended
PerplexityBot
Perplexity-User
Bytespider
Amazonbot
FacebookBot
meta-externalagent
meta-externalfetcher
cohere-ai
AI2Bot
Diffbot
omgili
YouBot
DuckAssistBot
MistralAI-User
PanguBot
Timpibot

Current robots.txt 301 bytes · sha256 4abc1bd3b4d0 · raw

User-agent: *
Disallow: /api/article-search
Disallow: /search-results/
Disallow: /video-search/
Disallow: /printer_friendly_story/
Disallow: /printer_friendly_wires/
Disallow: /wires/
Disallow: /xid


Sitemap: https://www.foxnews.com/sitemap.xml
Sitemap: https://www.foxnews.com/sitemap.xml?type=news

Change history

  1. initial snapshot
    • First snapshot of robots.txt archived

Editorial profile Content bias, reliability & geopolitical trust

Content biasRight CredibilityMixed reliability

Content bias (political lean) and credibility (factual track record) are third-party/aggregated assessments β€” source: AllSides/MBFC β€” and may be contested.