NeuralCrawl

🇺🇸 Fox News

foxnews.com · Publishers · rank #10 · Broadcast News · live robots.txt ↗

AI crawler access (latest snapshot, 21d ago)

⛔blocked restricted ✅allowed faded = inherited from the * wildcard group

✅GPTBot
✅ChatGPT-User
✅OAI-SearchBot
✅ClaudeBot
✅Claude-User
✅Claude-SearchBot
✅anthropic-ai
✅Claude-Web
✅CCBot
✅Google-Extended
✅Applebot-Extended
✅PerplexityBot
✅Perplexity-User
✅Bytespider
✅Amazonbot
✅FacebookBot
✅meta-externalagent
✅Meta-WebIndexer
✅meta-externalfetcher
✅cohere-ai
✅AI2Bot
✅Diffbot
✅omgili
✅YouBot
✅DuckAssistBot
✅MistralAI-User
✅PanguBot
✅Timpibot

Search engine crawler access

✅Googlebot
✅Googlebot-Image
✅Googlebot-News
✅Bingbot
✅Slurp
✅DuckDuckBot
✅Baiduspider
✅YandexBot
✅Applebot
✅Sogou
✅SeznamBot
✅PetalBot

Current robots.txt 301 bytes · sha256 4abc1bd3b4d0 · raw 

User-agent: *
Disallow: /api/article-search
Disallow: /search-results/
Disallow: /video-search/
Disallow: /printer_friendly_story/
Disallow: /printer_friendly_wires/
Disallow: /wires/
Disallow: /xid


Sitemap: https://www.foxnews.com/sitemap.xml
Sitemap: https://www.foxnews.com/sitemap.xml?type=news

Compare any two snapshots

Change history

  1. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  2. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  3. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  4. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  5. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  6. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  7. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  8. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  9. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  10. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  11. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  12. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  13. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  14. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  15. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  16. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  17. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  18. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  19. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  20. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  21. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  22. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  23. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  24. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  25. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  26. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  27. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  28. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  29. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  30. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  31. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  32. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  33. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  34. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  35. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  36. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  37. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  38. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  39. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  40. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  41. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  42. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  43. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  44. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  45. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  46. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  47. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  48. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  49. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  50. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  51. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  52. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  53. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  54. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  55. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  56. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  57. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  58. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  59. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  60. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  61. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  62. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  63. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  64. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  65. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  66. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  67. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  68. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  69. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  70. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  71. modified +3 −1 view diff → snapshot
    • Added Disallow: /api/article-search for *
    • Added Disallow: /xid for *
    • Removed sitemap https://video.foxnews.com/videositemap.xml
  72. modified +1 −3 view diff → snapshot
    • Removed Disallow: /api/article-search for *
    • Removed Disallow: /xid for *
    • Added sitemap https://video.foxnews.com/videositemap.xml
  73. initial snapshot
    • First snapshot of robots.txt archived

Editorial profile Content bias, reliability & geopolitical trust

Content biasRight CredibilityMixed reliability

Content bias (political lean) and credibility (factual track record) are third-party/aggregated assessments — source: AllSides/MBFC — and may be contested.