Cencora / robots.txt
← back to cencora.com · change detected 7h ago (2026-08-05T19:04:11Z) · modified +37 −1
What changed
- 🤖 AIAdded rules for AI crawler Amazonbot (Amazon)
- 🤖 AIAdded rules for AI crawler Applebot-Extended (Apple)
- 🤖 AIAdded rules for AI crawler CCBot (Common Crawl)
- 🤖 AIAdded rules for AI crawler ChatGPT-User (OpenAI)
- 🤖 AIAdded rules for AI crawler Claude-SearchBot (Anthropic)
- 🤖 AIAdded rules for AI crawler Claude-User (Anthropic)
- 🤖 AIAdded rules for AI crawler ClaudeBot (Anthropic)
- 🤖 AIAdded rules for AI crawler Google-Extended (Google)
- 🤖 AIAdded rules for AI crawler GPTBot (OpenAI)
- 🤖 AIAdded rules for AI crawler meta-externalagent (Meta)
- 🤖 AIAdded rules for AI crawler OAI-SearchBot (OpenAI)
- 🤖 AIAdded rules for AI crawler Perplexity-User (Perplexity)
- 🤖 AIAdded rules for AI crawler PerplexityBot (Perplexity)
Diff old (2026-06-25T22:38:12Z) → new (2026-08-05T19:04:11Z)
| @@ -1,3 +1,7 @@ | ||
| 1 | + # robots.txt for https://www.cencora.com | |
| 2 | + # Last reviewed: 2026-08-05 | |
| 3 | + | |
| 4 | + # ---- Utility bots ---- | |
| 1 | 5 | User-agent: DequeAxeMonitoring |
| 2 | 6 | Allow: / |
| 3 | 7 | |
| @@ -11,7 +15,21 @@ | ||
| 11 | 15 | User-agent: Semrushbot-SI |
| 12 | 16 | Allow: / |
| 13 | 17 | |
| 14 | - User-agent: * | |
| 18 | + # ---- AI crawlers ---- | |
| 19 | + User-agent: GPTBot | |
| 20 | + User-agent: OAI-SearchBot | |
| 21 | + User-agent: ChatGPT-User | |
| 22 | + User-agent: ClaudeBot | |
| 23 | + User-agent: Claude-SearchBot | |
| 24 | + User-agent: Claude-User | |
| 25 | + User-agent: PerplexityBot | |
| 26 | + User-agent: Perplexity-User | |
| 27 | + User-agent: Google-Extended | |
| 28 | + User-agent: Applebot-Extended | |
| 29 | + User-agent: meta-externalagent | |
| 30 | + User-agent: Amazonbot | |
| 31 | + User-agent: CCBot | |
| 32 | + Allow: / | |
| 15 | 33 | Disallow: /initiatives* |
| 16 | 34 | Disallow: /es/initiatives* |
| 17 | 35 | Disallow: /de/initiatives* |
| @@ -29,6 +47,24 @@ | ||
| 29 | 47 | Disallow: /Global-reach/Europe/United-Kingdom/Legal-regulatory-hub/BE-Force-terms-conditions |
| 30 | 48 | Disallow: /Global-reach/Europe/United-Kingdom/Legal-regulatory-hub/pharmacyone-brand-equalisation-terms |
| 31 | 49 | |
| 50 | + # ---- Default: all other bots ---- | |
| 51 | + User-agent: * | |
| 32 | 52 | Crawl-delay: 5 |
| 53 | + Disallow: /initiatives* | |
| 54 | + Disallow: /es/initiatives* | |
| 55 | + Disallow: /de/initiatives* | |
| 56 | + Disallow: /fr/initiatives* | |
| 57 | + Disallow: /ca/initiatives* | |
| 58 | + Disallow: /nl/initiatives* | |
| 59 | + Disallow: /no/initiatives* | |
| 60 | + Disallow: /it/initiatives* | |
| 61 | + Disallow: /ro/initiatives* | |
| 62 | + Disallow: /cz/initiatives* | |
| 63 | + Disallow: /tr/initiatives* | |
| 64 | + Disallow: /*thank-you | |
| 65 | + Disallow: /-/media/assets/corporate/aps/25-pah-10275-pharmacy-outlook-booklet-2025-02dec25-v5-fnl-digital.pdf | |
| 66 | + Disallow: /Global-reach/Europe/United-Kingdom/Legal-regulatory-hub/Almus-NetGen-Terms-Conditions | |
| 67 | + Disallow: /Global-reach/Europe/United-Kingdom/Legal-regulatory-hub/BE-Force-terms-conditions | |
| 68 | + Disallow: /Global-reach/Europe/United-Kingdom/Legal-regulatory-hub/pharmacyone-brand-equalisation-terms | |
| 33 | 69 | |
| 34 | 70 | Sitemap: https://www.cencora.com/sitemap.xml |