Regeneron Pharmaceuticals / robots.txt

What changed

• Response is a valid robots.txt again (was an HTML page)
• Added rules for user-agent ‘*’
• 🤖 AIAdded rules for AI crawler Amazonbot (Amazon)
• 🤖 AIAdded rules for AI crawler Applebot-Extended (Apple)
• 🤖 AIAdded rules for AI crawler Bytespider (ByteDance)
• 🤖 AIAdded rules for AI crawler CCBot (Common Crawl)
• 🤖 AIAdded rules for AI crawler ChatGPT-User (OpenAI)
• 🤖 AIAdded rules for AI crawler ClaudeBot (Anthropic)
• 🤖 AIAdded rules for AI crawler Diffbot (Diffbot)
• 🤖 AIAdded rules for AI crawler Google-Extended (Google)
• 🤖 AIAdded rules for AI crawler GPTBot (OpenAI)
• 🤖 AIAdded rules for AI crawler meta-externalagent (Meta)
• 🤖 AIAdded rules for AI crawler meta-externalfetcher (Meta)
• 🤖 AIAdded rules for AI crawler Perplexity-User (Perplexity)
• 🤖 AIAdded rules for AI crawler PerplexityBot (Perplexity)
• Added sitemap https://www.regeneron.com/images-sitemap.xml
• Added sitemap https://www.regeneron.com/sitemap.xml
• Added sitemap https://www.regeneron.com/video-sitemap.xml

Diff (no previous version) → (file removed)

No line-level differences (identical or both empty).