NeuralCrawl

IGN / robots.txt

← back to ign.com · change detected 25d ago (2026-10-12T17:28:08Z) · modified +5 −0

What changed

Diff old (2026-10-03T20:27:55Z) → new (2026-10-12T17:28:08Z)

@@ -26,6 +26,7 @@
2626  Disallow: /sugarfif.html
2727  Disallow: /wikis/*/edit
2828  Disallow: /wikis/*/compare/*
29+ Allow: /sponsored/
2930 
3031  #block crawlers
3132  User-agent: Amzn-SearchBot
@@ -42,6 +43,7 @@
4243  User-agent: opencode
4344  User-agent: TerraCotta
4445  User-agent: Trae
46+ Allow: /sponsored/
4547  Disallow: /
4648 
4749  #Block AI content scrapers
@@ -256,6 +258,7 @@
256258  Allow: /articles/ghost-of-yotei-review
257259  Allow: /articles/gta-6-release-date-about-18-months-behind-original-target-take-two-boss-reveals-suggesting-spring-2025-was-once-the-goal-internally
258260  Allow: /articles/hollow-knight-silksong-global-release-times-confirmed
261+ Allow: /articles/how-the-samsung-galaxy-z-fold8-has-been-optimised-for-large-screen-entertainment-multitasking-and-all-day-battery-life
259262  Allow: /articles/ign-awards-2025-winners
260263  Allow: /articles/jurassic-world-rebirth-review-scarlett-johansson-mahershala-ali
261264  Allow: /articles/metroid-prime-4-beyond-review
@@ -285,6 +288,7 @@
285288  Allow: /articles/video-game-release-dates-ps4-ps5-xbox-one-series-x-nintendo-switch
286289  Allow: /articles/what-to-expect-playstation-state-of-play-june-2026
287290  Allow: /articles/x-men-movies-in-order
291+ Allow: /articles/z-fold8-vs-previous-samsung-foldables-how-battery-performance-efficiency-and-display-experience-have-evolved
288292  Allow: /playlist/rchnemesis/lists/top-100-indie-games
289293  Allow: /wikis/007-first-light/Essential_Tips_and_Tricks
290294  Allow: /wikis/arc-raiders/Avian_Alarm_Project_Guide_and_How_to_Complete
@@ -351,6 +355,7 @@
351355  Disallow: /
352356 
353357  User-agent: DuckDuckBot
358+ Allow: /sponsored/
354359  Allow: /
355360 
356361  Sitemap: https://www.ign.com/rss/sitemap-articles.xml