NeuralCrawl — change feed

DSV dsv.com modified +16 −16 2mo ago

Formatting/comment-only change (no semantic difference)

view diff → snapshot

Bank Central Asia bca.co.id modified +1 −1 2mo ago

Added sitemap https://www.bca.co.id/sitemap.xml
Removed sitemap http://www.bca.co.id/sitemap.xml

view diff → snapshot

Colombia (Presidencia) presidencia.gov.co removed +0 −9 2mo ago

robots.txt is no longer served (HTTP 404)

Toronto Star thestar.com modified +16 −16 2mo ago

Formatting/comment-only change (no semantic difference)

view diff → snapshot

Figma Community figma.com modified +1 −0 2mo ago

🤖 AI Added Allow: /blog/sightlines-issue-no2/$ for CHATGPT-User
🤖 AI Added Allow: /blog/sightlines-issue-no2/$ for Claude-SearchBot
🤖 AI Added Allow: /blog/sightlines-issue-no2/$ for Claude-User
🤖 AI Added Allow: /blog/sightlines-issue-no2/$ for OAI-SearchBot
🤖 AI Added Allow: /blog/sightlines-issue-no2/$ for PerplexityBot

view diff → snapshot

Exa exa.ai modified +0 −1 2mo ago

Removed Disallow: /hero-variant-test for *

view diff → snapshot

Scale AI scale.com modified +1 −0 2mo ago

Added sitemap https://brand.scale.com/sitemap.xml

view diff → snapshot

eBay ebay.com modified +237 −1 2mo ago

Added rules for user-agent ‘Googlebot’

view diff → snapshot

IQVIA Holdings iqvia.com modified +2 −2 2mo ago

Added sitemap http://127.0.0.1/image-sitemap.xml
Added sitemap http://127.0.0.1/page-sitemap.xml
Removed sitemap http://www.iqvia.com/image-sitemap.xml
Removed sitemap http://www.iqvia.com/page-sitemap.xml

view diff → snapshot

DSV dsv.com modified +29 −29 2mo ago

Formatting/comment-only change (no semantic difference)

view diff → snapshot

Flipkart flipkart.com modified +1 −0 2mo ago

Added Disallow: */product-reviews/* for *

view diff → snapshot

Bank Central Asia bca.co.id modified +1 −1 2mo ago

Added sitemap http://www.bca.co.id/sitemap.xml
Removed sitemap https://www.bca.co.id/sitemap.xml

view diff → snapshot

Eli Lilly lilly.com modified +3 −0 2mo ago

Added sitemap https://www.lilly.com/jp/henzutsu-karuta/sitemap.xml
Added sitemap https://www.lilly.com/jp/locj/sitemap.xml
Added sitemap https://www.lilly.com/jp/sitemap.xml

view diff → snapshot

WordPress wordpress.com modified +1 −1 2mo ago

Formatting/comment-only change (no semantic difference)

view diff → snapshot

Toronto Star thestar.com modified +12 −12 2mo ago

Formatting/comment-only change (no semantic difference)

view diff → snapshot

ExtraHop extrahop.com modified +1 −1 2mo ago

Added sitemap https://www.extrahop.com/sitemap.xml
Removed sitemap https://www.extrahop.com/api/sitemap.xml

view diff → snapshot

Dell Technologies dell.com modified +3 −4 2mo ago

Added Disallow: /en-gb for *
Removed Disallow: /support/driver/*/api/drvdetails/swbsuccessors for *

view diff → snapshot

The Economist economist.com modified +16 −0 2mo ago

🤖 AI Added Allow: /syndication$ for ChatGPT-User
🤖 AI Added Allow: /syndication/ for ChatGPT-User
🤖 AI Added Allow: /syndication$ for Claude-SearchBot
🤖 AI Added Allow: /syndication/ for Claude-SearchBot
🤖 AI Added Allow: /syndication$ for Claude-User
🤖 AI Added Allow: /syndication/ for Claude-User
… 10 more change(s)

view diff → snapshot

Valero Energy valero.com modified +6 −0 2mo ago

Added Allow: /core/*.avif for *
Added Allow: /core/*.webp for *
Added Allow: /profiles/*.avif for *
Added Allow: /profiles/*.webp for *
Added Disallow: /index.php/search? for *
Added Disallow: /search? for *

view diff → snapshot

Vox vox.com modified +100 −0 2mo ago

Added Allow: /strategist for A6-Indexer
Added Allow: /strategist for AhrefsBot
🤖 AI Added Allow: /strategist for AI2Bot
Added Allow: /strategist for AI2Bot-DeepResearchEval
Added Allow: /strategist for Ai2Bot-Dolma
Added Allow: /strategist for aiHitBot
… 45 more change(s)

view diff → snapshot

Rubrik rubrik.com modified +1 −0 2mo ago

Added sitemap https://www.rubrik.com/sitemap-eds.xml

view diff → snapshot

Pinterest pinterest.com modified +11 −11 2mo ago

Formatting/comment-only change (no semantic difference)

view diff → snapshot

Adidas adidas.com modified +1 −1 2mo ago

Formatting/comment-only change (no semantic difference)

view diff → snapshot

IQVIA Holdings iqvia.com modified +2 −2 2mo ago

Added sitemap http://www.iqvia.com/image-sitemap.xml
Added sitemap http://www.iqvia.com/page-sitemap.xml
Removed sitemap http://127.0.0.1/image-sitemap.xml
Removed sitemap http://127.0.0.1/page-sitemap.xml

view diff → snapshot

DSV dsv.com modified +25 −25 2mo ago

Formatting/comment-only change (no semantic difference)

view diff → snapshot

Amgen amgen.com modified +2 −0 2mo ago

Formatting/comment-only change (no semantic difference)

view diff → snapshot

Bank Central Asia bca.co.id modified +1 −1 2mo ago

Added sitemap https://www.bca.co.id/sitemap.xml
Removed sitemap http://www.bca.co.id/sitemap.xml

view diff → snapshot

WordPress wordpress.com modified +1 −1 2mo ago

Formatting/comment-only change (no semantic difference)

view diff → snapshot

Colombia (Presidencia) presidencia.gov.co created +0 −0 2mo ago

Formatting/comment-only change (no semantic difference)

view diff → snapshot

Toronto Star thestar.com modified +15 −15 2mo ago

Formatting/comment-only change (no semantic difference)

view diff → snapshot

The Pulse