Cencora / robots.txt
← back to cencora.com · change detected 17d ago (2026-10-20T14:27:19Z) · modified +14 −2
What changed
- 🤖 AIAdded rules for AI crawler anthropic-ai (Anthropic)
- Added rules for user-agent ‘Applebot’
- Added rules for user-agent ‘Bingbot’
- 🤖 AIAdded rules for AI crawler Claude-Web (Anthropic)
- 🤖 AIAdded rules for AI crawler cohere-ai (Cohere)
- 🤖 AIAdded rules for AI crawler Diffbot (Diffbot)
- 🤖 AIAdded rules for AI crawler DuckAssistBot (DuckDuckGo)
- Added rules for user-agent ‘DuckDuckBot’
- Added rules for user-agent ‘Google-CloudVertexBot’
- Added rules for user-agent ‘Googlebot’
- 🤖 AIAdded rules for AI crawler meta-externalfetcher (Meta)
- 🤖 AIAdded rules for AI crawler MistralAI-User (Mistral AI)
Diff old (2026-08-05T19:04:11Z) → new (2026-10-20T14:27:19Z)
| @@ -1,5 +1,5 @@ | ||
| 1 | 1 | # robots.txt for https://www.cencora.com |
| 2 | - # Last reviewed: 2026-08-05 | |
| 2 | + # Last reviewed: 2026-10-20 | |
| 3 | 3 | |
| 4 | 4 | # ---- Utility bots ---- |
| 5 | 5 | User-agent: DequeAxeMonitoring |
| @@ -15,8 +15,12 @@ | ||
| 15 | 15 | User-agent: Semrushbot-SI |
| 16 | 16 | Allow: / |
| 17 | 17 | |
| 18 | - # ---- AI crawlers ---- | |
| 18 | + # ---- Search engines and AI crawlers ---- | |
| 19 | + User-agent: Applebot | |
| 20 | + USer-agent: DuckDuckBot | |
| 19 | 21 | User-agent: GPTBot |
| 22 | + User-agent: Bingbot | |
| 23 | + User-agent: Googlebot | |
| 20 | 24 | User-agent: OAI-SearchBot |
| 21 | 25 | User-agent: ChatGPT-User |
| 22 | 26 | User-agent: ClaudeBot |
| @@ -29,6 +33,14 @@ | ||
| 29 | 33 | User-agent: meta-externalagent |
| 30 | 34 | User-agent: Amazonbot |
| 31 | 35 | User-agent: CCBot |
| 36 | + User-agent: anthropic-ai | |
| 37 | + User-agent: Claude-Web | |
| 38 | + User-agent: cohere-ai | |
| 39 | + User-agent: Diffbot | |
| 40 | + User-agent: DuckAssistBot | |
| 41 | + User-agent: MistralAI-User | |
| 42 | + User-agent: Meta-ExternalFetcher | |
| 43 | + User-agent: Google-CloudVertexBot | |
| 32 | 44 | Allow: / |
| 33 | 45 | Disallow: /initiatives* |
| 34 | 46 | Disallow: /es/initiatives* |