NeuralCrawl

πŸ‡ΊπŸ‡Έ IGN

ign.com · Top 1000 websites · rank #104 · Web · live robots.txt ↗

AI crawler access (latest snapshot, 1h ago)

blocked restricted allowed faded = inherited from the * wildcard group

GPTBot
ChatGPT-User
OAI-SearchBot
ClaudeBot
Claude-User
Claude-SearchBot
anthropic-ai
Claude-Web
CCBot
Google-Extended
Applebot-Extended
PerplexityBot
Perplexity-User
Bytespider
Amazonbot
FacebookBot
meta-externalagent
meta-externalfetcher
cohere-ai
AI2Bot
Diffbot
omgili
YouBot
DuckAssistBot
MistralAI-User
PanguBot
Timpibot

Current robots.txt 11816 bytes · sha256 4e1229d06530 · raw

# Ziff Davis content is made available for your non-commercial use subject to our 
# Terms of Use here: https://www.ziffdavis.com/terms-of-use
# Use of any robot, crawler, or other tool to scrape, harvest, extract, or retrieve any content on
# this website using automated means is prohibited without written permission from Ziff Davis.
# Prohibited uses include but are not limited to:
# (1) text and data mining under Art. 4 of the EU Directive on Copyright in the Digital Single
# Market;
# (2) development or operation of artificial intelligence or machine learning software or
# databases, including by training, fine-tuning, embedding, and retrieval-augmented generation;
# (3) creating data sets containing our content or sharing it with others; and
# (4) any commercial purposes.
# Contact [email protected] for assistance.

User-agent: *
Disallow: /_views/
Disallow: /apiproxy
Disallow: */empty/
Disallow: /beta
Disallow: /blogs/
Disallow: /debug
Disallow: /pokedex/search
Disallow: /search
Disallow: /static/pv_candidate.html
Disallow: /stf/
Disallow: /sign-in
Disallow: /sugarfif.html
Disallow: /wikis/*/edit
Disallow: /wikis/*/compare/*

#block crawlers
User-agent: Amzn-SearchBot
User-agent: ApifyBot
User-agent: ApifyWebsiteContentCrawler
User-agent: AzureAI-SearchBot
User-agent: Brightbot
User-agent: ChatGPT-User
User-agent: ExaBot
User-agent: FirecrawlAgent
User-agent: GPTBot
User-agent: iAskBot
User-agent: OAI-SearchBot
User-agent: opencode
User-agent: TerraCotta
User-agent: Trae
Disallow: /

#Block AI content scrapers
User-agent: "008"
User-agent: AddSearchBot
User-agent: AI2Bot
User-agent: Ai2Bot-DeepResearchEval
User-agent: Ai2Bot-Dolma
User-agent: Amazonbot
User-agent: AmazonBuyForMe
User-agent: Anchor Browser
User-agent: anthropic-ai
User-agent: Applebot
User-agent: Applebot-extended
User-agent: atlassian-bot
User-agent: AutoRAG
User-agent: AwarioSmartBot
User-agent: Big Sur AI
User-agent: bigsur.ai
User-agent: Brandwatch
User-agent: Bravebot
User-agent: Bytespider
User-agent: CCBot
User-agent: Channel3Bot
User-agent: ChatGLM-Spider
User-agent: Claude-SearchBot
User-agent: Claude-User
User-agent: Claude-Web
User-agent: ClaudeBot
User-agent: cohere-ai
User-agent: cohere-training-data-crawler
User-agent: Cotoyogi
User-agent: Crawl4AI
User-agent: Datenbank Crawler
User-agent: DDM*
User-agent: DDM-DCipher*
User-agent: DDM-DCipher/1.0.7
User-agent: DeepSeekBot
User-agent: Devin
User-agent: Diffbot
User-agent: Direqt Anomura
User-agent: DuckAssistBot
User-agent: Echobot Bot
User-agent: FacebookBot
User-agent: Factset_spyderbot
User-agent: HTTrack
User-agent: iAsk
User-agent: iaskspider
User-agent: ICC Crawler
User-agent: ICC-Crawler
User-agent: ImageSiftBot
User-agent: imageSpider
User-agent: Kangaroo Bot
User-agent: KlaviyoAIBot
User-agent: Kunato
User-agent: laion-huggingface-processor
User-agent: LCC
User-agent: LINER Bot
User-agent: LinerBot
User-agent: LinkupBot
User-agent: Manus-User
User-agent: meta-externalagent
User-agent: meta-externalfetcher
User-agent: meta-webindexer
User-agent: mistral.ai
User-agent: MistralAI-User
User-agent: netEstate Imprint Crawler
User-agent: NovaAct
User-agent: Novellum AI Crawl
User-agent: Nutch
User-agent: Offline Explorer
User-agent: omgili
User-agent: PanguBot
User-agent: PeopleInc-DCipher-Scraper/1.1.0
User-agent: Perplexity-User
User-agent: PerplexityBot
User-agent: PetalBot
User-agent: Poggio-Citations
User-agent: QualifiedBot
User-agent: SBIntuitionsBot
User-agent: Scrapy
User-agent: SeekrBot
User-agent: SemrushBot-OCOB
User-agent: SemrushBotSwa
User-agent: ShapBot
User-agent: Spider
User-agent: TavilyBot
User-agent: Timpibot
User-agent: TwinAgent
User-agent: Velen Crawler
User-agent: VelenPublicWebCrawler
User-agent: Webzio-Extended
User-agent: Wrtn
User-agent: YouBot
User-agent: ZanistaBot
Allow: /articles/experimenting-with-galaxy-ai-camera-features-like-night-mode-generative-edit-and-more
Allow: /articles/how-ai-assisted-gaming-on-the-galaxy-s25-means-extended-play-time-and-optimized-performance
Allow: /articles/007-first-light-review
Allow: /articles/1080p-at-30fps-valve-steam-machines-verified-program
Allow: /articles/alien-earth-review
Allow: /articles/avatar-fire-and-ash-is-now-available-to-watch-at-home
Allow: /articles/battlefield-6-global-release-times-confirmed
Allow: /articles/battlefield-6-review-multiplayer
Allow: /articles/best-assassins-creed-games
Allow: /articles/best-games-like-elder-scrolls-5-skyrim
Allow: /articles/best-gaming-headset
Allow: /articles/best-gaming-keyboard
Allow: /articles/best-gaming-laptop
Allow: /articles/best-gaming-monitor
Allow: /articles/best-graphics-card
Allow: /articles/best-handheld-gaming-pc
Allow: /articles/best-horror-games
Allow: /articles/best-lego-games
Allow: /articles/best-pc-controller
Allow: /articles/best-playstation-portal-accessories
Allow: /articles/best-ps5-accessories
Allow: /articles/best-samsung-galaxy-s26-cases
Allow: /articles/best-tv-for-gaming
Allow: /articles/best-xbox-series-x-games
Allow: /articles/blox-fruits-codes
Allow: /articles/blue-lock-rivals-codes
Allow: /articles/borderlands-4-review
Allow: /articles/call-of-duty-black-ops-7-campaign-review
Allow: /articles/conjuring-movies-in-order
Allow: /articles/crimson-desert-global-release-time-confirmed-as-preload-goes-live
Allow: /articles/crimson-desert-review
Allow: /articles/disney-debuts-new-avengers-doomsday-trailer-at-cinemacon-2026
Allow: /articles/dress-to-impress-codes
Allow: /articles/ghost-of-yotei-review
Allow: /articles/gta-6-release-date-about-18-months-behind-original-target-take-two-boss-reveals-suggesting-spring-2025-was-once-the-goal-internally
Allow: /articles/hollow-knight-silksong-global-release-times-confirmed
Allow: /articles/ign-awards-2025-winners
Allow: /articles/jurassic-world-rebirth-review-scarlett-johansson-mahershala-ali
Allow: /articles/metroid-prime-4-beyond-review
Allow: /articles/new-xbox-ceo-comments-on-game-pass
Allow: /articles/nintendo-fans-demand-tariff-refunds-returned-to-customers
Allow: /articles/nintendo-fans-surprised-to-find-switch-2-edition-game-cartridges-also-work-on-switch-1
Allow: /articles/nintendo-is-absolutely-furious-former-employee-says-mario-zelda-developer-in-uncharted-territory-following-major-leaks
Allow: /articles/nintendo-switch-2-price-hike-where-to-buy
Allow: /articles/one-piece-series-movies-in-order
Allow: /articles/phantom-blade-zero-is-a-delightfully-punishing-action-rpg
Allow: /articles/plants-vs-brainrots-codes
Allow: /articles/pokemon-fan-claims-world-first-completion-of-resident-evil-requiem-final-puzzle-but-claim-theyre-unsure-exactly-how-they-did-it
Allow: /articles/resident-evil-requiem-review
Allow: /articles/silent-hill-f-review
Allow: /articles/sony-announces-price-rises-for-ps5-ps5-pro-and-playstation-portal-blames-continued-pressures-in-the-global-economic-landscape
Allow: /articles/sony-confirms-playstation-plus-monthly-games-for-april-2026
Allow: /articles/steam-machine-hands-on-first-impressions
Allow: /articles/stranger-things-creators-promise-say-theyll-reveal-elevens-fate-in-20-years
Allow: /articles/superman-2025-review-james-gunn-david-corenswet-rachel-brosnahan-nicholas-hoult
Allow: /articles/the-10-best-pokemon-video-games
Allow: /articles/the-best-100-video-games-of-all-time
Allow: /articles/the-best-gaming-chairs
Allow: /articles/the-best-ps5-games
Allow: /articles/the-fate-anime-series-in-order
Allow: /articles/the-forge-codes
Allow: /articles/the-last-of-us-2-safe-combinations-and-codes
Allow: /articles/video-game-release-dates-ps4-ps5-xbox-one-series-x-nintendo-switch
Allow: /articles/what-to-expect-playstation-state-of-play-june-2026
Allow: /articles/x-men-movies-in-order
Allow: /playlist/rchnemesis/lists/top-100-indie-games
Allow: /wikis/007-first-light/Essential_Tips_and_Tricks
Allow: /wikis/arc-raiders/Avian_Alarm_Project_Guide_and_How_to_Complete
Allow: /wikis/avatar-frontiers-of-pandora/All_Sarentu_Totem_Locations
Allow: /wikis/baldurs-gate-3/The_Blood_of_Lathander
Allow: /wikis/blue-prince/How_to_Solve_the_Chess_Piece_Puzzle
Allow: /wikis/borderlands-4/Borderlands_4_SHiFT_Codes
Allow: /wikis/call-of-duty-black-ops-7/Walkthrough
Allow: /wikis/clair-obscur-expedition-33/All_Endings_and_How_to_Get_Them
Allow: /wikis/clair-obscur-expedition-33/Walkthrough
Allow: /wikis/crimson-desert/How_to_Get_the_Frostcursed_Armor_Set_-_All_Locations
Allow: /wikis/crimson-desert/Smoking_Lands_Desert_Mouth_Maze_Puzzle_Solution
Allow: /wikis/crimson-desert/Walkthrough
Allow: /wikis/cyberpunk-2077/Panam_Romance_Guide
Allow: /wikis/elden-ring/Millicent_Questline
Allow: /wikis/elden-ring/Ranni_The_Witch_Questline
Allow: /wikis/fallout-4/Companion_Perks
Allow: /wikis/final-fantasy-7-rebirth/FF7_Rebirth_Walkthrough
Allow: /wikis/forza-horizon-6/Barn_Finds
Allow: /wikis/ghost-of-yotei/Trophy_List_and_Guide
Allow: /wikis/grand-theft-auto-san-andreas/Cheat_Codes_and_Secrets
Allow: /wikis/grow-a-garden/All_Bizzy_Bear_Honey_Crafter_Recipe_Quests_and_Rewards
Allow: /wikis/grow-a-garden/All_Fruit_Plants_and_How_to_Get_Them
Allow: /wikis/grow-a-garden/All_Vegetables_and_How_to_Get_Them
Allow: /wikis/grow-a-garden/How_to_Make_a_Waffle
Allow: /wikis/gta-5/All_GTA_5_Cell_Phone_Cheat_codes
Allow: /wikis/gta-5/All_GTA_5_Cheat_Codes_and_Secrets_For_PC_and_Console
Allow: /wikis/gta-5/GTA_Online_Weekly_Updates
Allow: /wikis/hogwarts-legacy/All_Hogwarts_Secrets_Solutions
Allow: /wikis/hogwarts-legacy/All_Merlin_Trials_Solutions
Allow: /wikis/hogwarts-legacy/Demiguise_Statues_Locations
Allow: /wikis/hollow-knight-silksong/Pale_Oil_Locations
Allow: /wikis/hollow-knight-silksong/Widow_Boss_Guide
Allow: /wikis/hollow-knight/walkthrough
Allow: /wikis/kingdom-hearts-hd-15-remix/Kingdom_Hearts_Final_Mix_-_Walkthrough
Allow: /wikis/la-noire/Walkthrough
Allow: /wikis/mario-kart-world/All_Characters,_NPC_Drivers,_and_Outfits_List
Allow: /wikis/neverness-to-everness/All_NTE_Codes_for_June_2026
Allow: /wikis/paralives/Paralives_Cheats_List_and_Console_Commands
Allow: /wikis/pokemon-firered-leafgreen-version/Best_Starter_to_Choose
Allow: /wikis/pokemon-firered-leafgreen-version/Rock_Tunnel
Allow: /wikis/pokemon-firered-leafgreen-version/Route_22
Allow: /wikis/pokemon-legends-z-a/All_Mega_Pokemon
Allow: /wikis/re2-remake/All_Safe_and_Lock_Combinations,_Codes,_and_Solutions
Allow: /wikis/red-dead-redemption-2/Gold_Bar_Locations_and_Where_to_Sell_Them
Allow: /wikis/red-dead-redemption-2/Horses_Guide_-_Locations,_Stats,_Bonding,_Taming,_and_Breeds
Allow: /wikis/red-dead-redemption-2/Legendary_Animals_and_Fish_Locations_and_Map
Allow: /wikis/resident-evil-requiem/File_Locations
Allow: /wikis/resident-evil-requiem/Walkthrough
Allow: /wikis/silent-hill-2-remake/Endings
Allow: /wikis/silent-hill-f/Dark_Shrine_Finale_Walkthrough
Allow: /wikis/silent-hill-f/Field_Puzzle_Guide
Allow: /wikis/silent-hill-f/Light_Puzzle_Guide
Allow: /wikis/silent-hill-f/Walkthrough
Allow: /wikis/slime-rng-roblox/Slime_RNG_Codes
Allow: /wikis/stardew-valley/Item_Codes_for_Spawning_Cheat
Allow: /wikis/subnautica-2/Walkthrough_Hub
Allow: /wikis/the-elder-scrolls-5-skyrim/PC_Console_Command_Cheats
Allow: /wikis/the-forge/How_to_Reroll_Race
Allow: /wikis/the-legend-of-zelda-breath-of-the-wild/Shrines
Allow: /wikis/the-legend-of-zelda-tears-of-the-kingdom/All_Shrine_Locations_and_Solutions
Allow: /wikis/vv-ultimatum-roblox/VV_ULTIMATUM_Codes
Disallow: /

User-agent: DuckDuckBot
Allow: /

Sitemap: https://www.ign.com/rss/sitemap-articles.xml
Sitemap: https://www.ign.com/rss/sitemap-authors.xml
Sitemap: https://www.ign.com/rss/sitemap-objects.xml
Sitemap: https://www.ign.com/rss/sitemap-videos.xml
Sitemap: https://www.ign.com/rss/sitemap-wikis.xml
Sitemap: https://www.ign.com/rss/news/sitemap
Sitemap: https://www.ign.com/rss/videos/sitemap-index-recently-updated
Sitemap: https://www.ign.com/rss/wikis/sitemap-maps

Change history

  1. initial snapshot
    • First snapshot of robots.txt archived