Intel / robots.txt
← back to intel.com · change detected 11d ago (2026-09-07T23:51:50Z) · modified +23 −1
What changed
- 🤖 AIAdded rules for AI crawler anthropic-ai (Anthropic)
- 🤖 AIAdded rules for AI crawler ChatGPT-User (OpenAI)
- 🤖 AIAdded rules for AI crawler ClaudeBot (Anthropic)
- 🤖 AIAdded rules for AI crawler Google-Extended (Google)
- 🤖 AIAdded rules for AI crawler GPTBot (OpenAI)
- 🤖 AIAdded rules for AI crawler PerplexityBot (Perplexity)
- Added Allow: /llms-full.txt for *
- Added Allow: /llms.txt for *
Diff old (2026-06-20T01:10:30Z) → new (2026-09-07T23:51:50Z)
| @@ -1,7 +1,7 @@ | ||
| 1 | 1 | # robots.txt exclusion for www.intel.com/ - US |
| 2 | 2 | |
| 3 | 3 | User-agent: OmtrBot/1.0 |
| 4 | - Allow: /content/www/us/en/products/devices-systems/ | |
| 4 | + Allow: /content/www/us/en/products/devices-systems/ | |
| 5 | 5 | Allow: /content/dam/products/ |
| 6 | 6 | Allow: /libs/apps/intel/services/productseedlist |
| 7 | 7 | Disallow: /cgi |
| @@ -38,7 +38,29 @@ | ||
| 38 | 38 | Disallow: /*/wp-01288-intel-fpgas-accelerate-intel-xeon-scalable*.pdf$ |
| 39 | 39 | Disallow: /content/dam/www/secure/ |
| 40 | 40 | |
| 41 | + # GEO: Allow all major AI crawlers — supports brand citation in AI-generated answers | |
| 42 | + | |
| 43 | + User-agent: GPTBot | |
| 44 | + Allow: / | |
| 45 | + | |
| 46 | + User-agent: ChatGPT-User | |
| 47 | + Allow: / | |
| 48 | + | |
| 49 | + User-agent: Google-Extended | |
| 50 | + Allow: / | |
| 51 | + | |
| 52 | + User-agent: anthropic-ai | |
| 53 | + Allow: / | |
| 54 | + | |
| 55 | + User-agent: ClaudeBot | |
| 56 | + Allow: / | |
| 57 | + | |
| 58 | + User-agent: PerplexityBot | |
| 59 | + Allow: / | |
| 60 | + | |
| 41 | 61 | User-agent: * |
| 62 | + Allow: /llms.txt | |
| 63 | + Allow: /llms-full.txt | |
| 42 | 64 | Allow: /content/dam/products/catalog/ |
| 43 | 65 | Allow: /content/dam/products/hero/ |
| 44 | 66 | Disallow: /cgi |