🇺🇸 ESPN

AI crawler access (latest snapshot, 1h ago)

⛔blocked restricted ✅allowed faded = inherited from the * wildcard group

Current robots.txt 3223 bytes · sha256 69e57dadcb6f · raw

# robots.txt for www.espn.com

User-agent: claritybot
Disallow: /

User-agent: GPTBot
Disallow: /

User-agent: Google-Extended
Disallow: /

User-agent: CCBot
Disallow: /

User-agent: ChatGPT-User
Disallow: /

User-agent: anthropic-ai
Disallow: /

User-agent: Omgilibot
Disallow: /

User-agent: Omgili
Disallow: /

User-agent: FacebookBot
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: *
Disallow: */admin/
Disallow: */boxscore?
Disallow: */calendar/
Disallow: */cat/
Disallow: */conversation?
Disallow: */conversation/
Disallow: */conversations?
Disallow: */conversations/
Disallow: */databaseresults/
Disallow: */date/
Disallow: */deportes/
Disallow: */flash/
Disallow: */login?
Disallow: */login/
Disallow: */order/
Disallow: */photoId/
Disallow: */playbyplay?
Disallow: /watch/*?*lang=
Disallow: /watch/*?*rb=
Disallow: /watch/*?*country=
Disallow: /watch/*?*search=
Disallow: /watch/*?*device=
Disallow: /watch/*?*partner=
Disallow: /watch/*?*redirected=
Disallow: /watch/_/redirected/*
Disallow: /watch/*?*om-navmethod=
Disallow: */print?id
Disallow: */printschedule/
Disallow: */season/200
Disallow: */split/
Disallow: */start/
Disallow: */swf/
Disallow: */util/
Disallow: */webslices/
Disallow: */year/200
Disallow: /*?ex_cid=
Disallow: /*?innings=
Disallow: /*?user=
Disallow: /*&action=login
Disallow: /*&full=
Disallow: /*&order=
Disallow: /*&photoId=
Disallow: /*&sort=*
Disallow: /ad/
Disallow: /broadband/
Disallow: /cgi
Disallow: /community/
Disallow: /composer/
Disallow: /contests/
Disallow: /espn/now
Disallow: /espnradio/podcast/feeds/easports/
Disallow: /index?sport=*&topId=*
Disallow: /index?sport=*&type=replay
Disallow: /members/
Disallow: /personalization/
Disallow: /sports*/index?topId=*
Disallow: /travel/passport/
Disallow: /video/clipDeportes
Disallow: /video/search
Disallow: /nba/tradeMachine/?tradeId=*
Disallow: */wire?
Disallow: /load_video_player/*
Disallow: /football/conversation?gameId=*
Disallow: */channels/*
Disallow: */crossdomain/*
Disallow: */makecorsrequest/*
Disallow: */utils/*
Disallow: */strategies/*
Disallow: */child/*
Disallow: */message/*
Disallow: */units/*
Disallow: */om-navmethod/*
Disallow: */null
Disallow: */null/*
Disallow: */undefined
Disallow: */undefined/*
Disallow: *ESPNPlusAdobeTracking/
Disallow: */modifier/*
Disallow: /*&xhr=1
Disallow: /*&wsVar=*
Disallow: /*&_slug_=*
Disallow: /*.com
Disallow: /*.net
Disallow: /*.org
Disallow: /*.co.uk
Disallow: */platform/*
Disallow: /*&platform=
Disallow: /*?platform=
Disallow: /football/matchstats
Disallow: /football/lineups
Disallow: /football/commentary
Disallow: /soccer/matchstats
Disallow: /football/video
Disallow: /soccer/lineups
Disallow: /soccer/commentary
Disallow: /soccer/video
Disallow: //*
Disallow: /*?*::*
Disallow: /rugby?*
Disallow: /jayski/tracks/archive/story/_/id/*/*/*
Disallow: */baltimore-ravens-head-coach-john-harbaugh-clocks-long-hours-prep-game-day-espn-magazine
Disallow: */kansas-city-royals-manager-ned-yost-lightning-rod-criticism
Disallow: */wwe-aj-styles-retains-top-spot-narrowest-margins-kevin-owens-john-cena
Allow: */platform/amp*
Allow: /*&platform=amp
Allow: /*?platform=amp
Allow: /.well-known/amphtml/apikey.pub
Sitemap: https://www.espn.com/sitemap.xml

Change history

1. initial 1h ago snapshot

   • First snapshot of robots.txt archived