NeuralCrawl

Instagram / robots.txt

← back to instagram.com · change detected 1d ago (2026-11-05T19:53:47Z) · modified +3 −0

What changed

Diff old (2026-10-29T17:28:06Z) → new (2026-11-05T19:53:47Z)

@@ -5,6 +5,9 @@
55  # All authorized user-agents listed on this page must comply with Meta’s
66  # Automated Data Collection Terms available at:
77  # https://www.facebook.com/legal/automated_data_collection_terms
8+ User-agent: __usage_ai-training
9+ Disallow: /
10+
811  User-agent: Amazonbot
912  Disallow: /
1013