Arrow Electronics / robots.txt
← back to arrow.com · change detected 25d ago (2026-10-13T05:44:23Z) · modified +5 −42
What changed
- Added sitemap https://www.arrow.com/commerce/sitemap/categories/sitemap-categories-index.xml
- Added sitemap https://www.arrow.com/commerce/sitemap/manufacturers/sitemap-manufacturers-index.xml
- Added sitemap https://www.arrow.com/commerce/sitemap/products/sitemap-products-index.xml
- Added sitemap https://www.arrow.com/commerce/sitemap/referencedesigns/sitemap-referencedesign-index.xml
- Added sitemap https://www.arrow.com/sitemap.xml
- Removed sitemap https://www.arrow.com/ais/msembedded/sitemap_index.xml
- Removed sitemap https://www.arrow.com/ais/sitemap_index.xml
- Removed sitemap https://www.arrow.com/digital-content/sitemap-index.xml
- Removed sitemap https://www.arrow.com/ecs/at/sitemap
- Removed sitemap https://www.arrow.com/ecs/au/sitemap
- Removed sitemap https://www.arrow.com/ecs/ch-de/sitemap/
- Removed sitemap https://www.arrow.com/ecs/ch-en/sitemap/
- Removed sitemap https://www.arrow.com/ecs/cz/sitemap/
- Removed sitemap https://www.arrow.com/ecs/de/sitemap/
- Removed sitemap https://www.arrow.com/ecs/dk/sitemap/
- Removed sitemap https://www.arrow.com/ecs/ee/sitemap
- Removed sitemap https://www.arrow.com/ecs/es/sitemap/
- Removed sitemap https://www.arrow.com/ecs/fi/sitemap/
- Removed sitemap https://www.arrow.com/ecs/fr/sitemap/
- Removed sitemap https://www.arrow.com/ecs/ie/sitemap/
- Removed sitemap https://www.arrow.com/ecs/il/sitemap
- Removed sitemap https://www.arrow.com/ecs/it/sitemap/
- Removed sitemap https://www.arrow.com/ecs/ma/sitemap
- Removed sitemap https://www.arrow.com/ecs/na/sitemap
- Removed sitemap https://www.arrow.com/ecs/nl/sitemap/
- Removed sitemap https://www.arrow.com/ecs/no/sitemap/
- Removed sitemap https://www.arrow.com/ecs/nz/sitemap
- Removed sitemap https://www.arrow.com/ecs/pt/sitemap/
- Removed sitemap https://www.arrow.com/ecs/se/sitemap/
- Removed sitemap https://www.arrow.com/ecs/uk/sitemap/
- Removed sitemap https://www.arrow.com/s-tech/category-sitemap.xml
- Removed sitemap https://www.arrow.com/s-tech/page-sitemap.xml
- Removed sitemap https://www.arrow.com/s-tech/post-sitemap.xml
- Removed sitemap https://www.arrow.com/s-tech/post_tag-sitemap.xml
Diff old (2026-10-11T23:13:09Z) → new (2026-10-13T05:44:23Z)
| @@ -9,15 +9,8 @@ | ||
| 9 | 9 | Disallow: */design-center/my-projects |
| 10 | 10 | Disallow: /technical-content/ |
| 11 | 11 | |
| 12 | - #Commenting all below lines to allow crawling of URLs with query paramters | |
| 13 | - #Disallow: *?region* | |
| 14 | - #Disallow: *utm_source=* | |
| 15 | - #Disallow: *%3Fregion* | |
| 16 | - | |
| 17 | 12 | Disallow: */datasheets/ |
| 18 | 13 | Disallow: */login* |
| 19 | - #Disallow: */arrow-dxp/language-masters/ | |
| 20 | - #Disallow: /content/arrow-dxp/ | |
| 21 | 14 | |
| 22 | 15 | Disallow: /~/media/technical-content/ |
| 23 | 16 | Disallow: */my-account |
| @@ -45,7 +38,6 @@ | ||
| 45 | 38 | Disallow: / |
| 46 | 39 | Allow: /ais/* |
| 47 | 40 | Allow: /globalecs/* |
| 48 | - | |
| 49 | 41 | User-agent: ClaudeBot |
| 50 | 42 | Disallow: / |
| 51 | 43 | Allow: /ais/* |
| @@ -75,42 +67,13 @@ | ||
| 75 | 67 | Disallow: / |
| 76 | 68 | Allow: /ais/* |
| 77 | 69 | Allow: /globalecs/* |
| 78 | - | |
| 79 | - Sitemap: https://www.arrow.com/digital-content/sitemap-index.xml | |
| 80 | 70 | |
| 81 | - Sitemap: https://www.arrow.com/s-tech/post-sitemap.xml | |
| 82 | - Sitemap: https://www.arrow.com/s-tech/page-sitemap.xml | |
| 83 | - Sitemap: https://www.arrow.com/s-tech/category-sitemap.xml | |
| 84 | - Sitemap: https://www.arrow.com/s-tech/post_tag-sitemap.xml | |
| 71 | + Sitemap: https://www.arrow.com/sitemap.xml | |
| 85 | 72 | |
| 86 | - Sitemap: https://www.arrow.com/ecs/ma/sitemap | |
| 87 | - Sitemap: https://www.arrow.com/ecs/il/sitemap | |
| 88 | - Sitemap: https://www.arrow.com/ecs/na/sitemap | |
| 89 | - Sitemap: https://www.arrow.com/ecs/au/sitemap | |
| 73 | + Sitemap: https://www.arrow.com/commerce/sitemap/products/sitemap-products-index.xml | |
| 90 | 74 | |
| 91 | - Sitemap: https://www.arrow.com/ecs/nz/sitemap | |
| 92 | - Sitemap: https://www.arrow.com/ecs/at/sitemap | |
| 93 | - Sitemap: https://www.arrow.com/ecs/ee/sitemap | |
| 75 | + Sitemap: https://www.arrow.com/commerce/sitemap/categories/sitemap-categories-index.xml | |
| 94 | 76 | |
| 95 | - Sitemap: https://www.arrow.com/ecs/cz/sitemap/ | |
| 96 | - Sitemap: https://www.arrow.com/ecs/dk/sitemap/ | |
| 97 | - Sitemap: https://www.arrow.com/ecs/de/sitemap/ | |
| 77 | + Sitemap: https://www.arrow.com/commerce/sitemap/manufacturers/sitemap-manufacturers-index.xml | |
| 98 | 78 | |
| 99 | - Sitemap: https://www.arrow.com/ecs/fi/sitemap/ | |
| 100 | - Sitemap: https://www.arrow.com/ecs/fr/sitemap/ | |
| 101 | - Sitemap: https://www.arrow.com/ecs/it/sitemap/ | |
| 102 | - | |
| 103 | - Sitemap: https://www.arrow.com/ecs/ie/sitemap/ | |
| 104 | - Sitemap: https://www.arrow.com/ecs/nl/sitemap/ | |
| 105 | - Sitemap: https://www.arrow.com/ecs/no/sitemap/ | |
| 106 | - | |
| 107 | - Sitemap: https://www.arrow.com/ecs/pt/sitemap/ | |
| 108 | - Sitemap: https://www.arrow.com/ecs/es/sitemap/ | |
| 109 | - Sitemap: https://www.arrow.com/ecs/se/sitemap/ | |
| 110 | - | |
| 111 | - Sitemap: https://www.arrow.com/ecs/ch-en/sitemap/ | |
| 112 | - Sitemap: https://www.arrow.com/ecs/ch-de/sitemap/ | |
| 113 | - Sitemap: https://www.arrow.com/ecs/uk/sitemap/ | |
| 114 | - | |
| 115 | - Sitemap: https://www.arrow.com/ais/sitemap_index.xml | |
| 116 | - Sitemap: https://www.arrow.com/ais/msembedded/sitemap_index.xml | |
| 79 | + Sitemap: https://www.arrow.com/commerce/sitemap/referencedesigns/sitemap-referencedesign-index.xml |