Arrow Electronics / robots.txt
← back to arrow.com · change detected 30d ago (2026-10-08T10:00:34Z) · created +13 −0
What changed
- 🤖 AIAdded Allow: /globalecs/* for anthropic-ai
- 🤖 AIAdded Allow: /globalecs/* for CCBot
- 🤖 AIAdded Allow: /globalecs/* for ChatGPT-User
- 🤖 AIAdded Allow: /globalecs/* for ClaudeBot
- 🤖 AIAdded Allow: /globalecs/* for Google-Extended
- 🤖 AIAdded Allow: /globalecs/* for GPTBot
- 🤖 AIAdded Allow: /globalecs/* for PerplexityBot
Diff old (2026-09-30T01:00:52Z) → new (2026-10-08T10:00:34Z)
| @@ -44,24 +44,37 @@ | ||
| 44 | 44 | User-agent: anthropic-ai |
| 45 | 45 | Disallow: / |
| 46 | 46 | Allow: /ais/* |
| 47 | + Allow: /globalecs/* | |
| 48 | + | |
| 47 | 49 | User-agent: ClaudeBot |
| 48 | 50 | Disallow: / |
| 49 | 51 | Allow: /ais/* |
| 52 | + Allow: /globalecs/* | |
| 53 | + | |
| 50 | 54 | User-agent: Google-Extended |
| 51 | 55 | Disallow: / |
| 52 | 56 | Allow: /ais/* |
| 57 | + Allow: /globalecs/* | |
| 58 | + | |
| 53 | 59 | User-agent: GPTBot |
| 54 | 60 | Disallow: / |
| 55 | 61 | Allow: /ais/* |
| 62 | + Allow: /globalecs/* | |
| 63 | + | |
| 56 | 64 | User-agent: ChatGPT-User |
| 57 | 65 | Disallow: / |
| 58 | 66 | Allow: /ais/* |
| 67 | + Allow: /globalecs/* | |
| 68 | + | |
| 59 | 69 | User-agent: CCBot |
| 60 | 70 | Disallow: / |
| 61 | 71 | Allow: /ais/* |
| 72 | + Allow: /globalecs/* | |
| 73 | + | |
| 62 | 74 | User-agent: PerplexityBot |
| 63 | 75 | Disallow: / |
| 64 | 76 | Allow: /ais/* |
| 77 | + Allow: /globalecs/* | |
| 65 | 78 | |
| 66 | 79 | Sitemap: https://www.arrow.com/digital-content/sitemap-index.xml |
| 67 | 80 |