Best Buy / robots.txt
← back to bestbuy.com · change detected 1d ago (2026-09-17T15:11:14Z) · modified +0 −11
What changed
- Removed Disallow: */site/searchpage.jsp?st=* for *
- Removed Disallow: *?cp=*&id=*&st=* for *
- Removed Disallow: *?id*_facet*&st=* for *
- Removed Disallow: *?id*_facet*_facet*&st=* for *
- 🤖 AIRemoved Disallow: *?cp=*&id=*&st=* for AmazonBot
- 🤖 AIRemoved Disallow: *?id*_facet*&st=* for AmazonBot
- 🤖 AIRemoved Disallow: *?id*_facet*_facet*&st=* for AmazonBot
- Removed Disallow: *?cp=*&id=*&st=* for Applebot
- Removed Disallow: *?id*_facet*&st=* for Applebot
- Removed Disallow: *?id*_facet*_facet*&st=* for Applebot
- Removed Disallow: */site/searchpage.jsp?st=* for Bingbot
- Removed Disallow: *?cp=*&id=*&st=* for Bingbot
- Removed Disallow: *?id*_facet*&st=* for Bingbot
- Removed Disallow: *?id*_facet*_facet*&st=* for Bingbot
- 🤖 AIRemoved Disallow: *?cp=*&id=*&st=* for ChatGPT-User
- 🤖 AIRemoved Disallow: *?id*_facet*&st=* for ChatGPT-User
- 🤖 AIRemoved Disallow: *?id*_facet*_facet*&st=* for ChatGPT-User
- 🤖 AIRemoved Disallow: *?cp=*&id=*&st=* for Claude-User
- 🤖 AIRemoved Disallow: *?id*_facet*&st=* for Claude-User
- 🤖 AIRemoved Disallow: *?id*_facet*_facet*&st=* for Claude-User
- 🤖 AIRemoved Disallow: *?cp=*&id=*&st=* for Claudebot
- 🤖 AIRemoved Disallow: *?id*_facet*&st=* for Claudebot
- 🤖 AIRemoved Disallow: *?id*_facet*_facet*&st=* for Claudebot
- Removed Disallow: */site/searchpage.jsp?st=* for DuckDuckBot
- Removed Disallow: *?cp=*&id=*&st=* for DuckDuckBot
- Removed Disallow: *?id*_facet*&st=* for DuckDuckBot
- Removed Disallow: *?id*_facet*_facet*&st=* for DuckDuckBot
- Removed Disallow: *?cp=*&id=*&st=* for Gemeni-AI
- Removed Disallow: *?id*_facet*&st=* for Gemeni-AI
- Removed Disallow: *?id*_facet*_facet*&st=* for Gemeni-AI
- Removed Disallow: *?cp=*&id=*&st=* for Gemini-Deep-Research
- Removed Disallow: *?id*_facet*&st=* for Gemini-Deep-Research
- Removed Disallow: *?id*_facet*_facet*&st=* for Gemini-Deep-Research
- 🤖 AIRemoved Disallow: *?cp=*&id=*&st=* for Google-Extended
- 🤖 AIRemoved Disallow: *?id*_facet*&st=* for Google-Extended
- 🤖 AIRemoved Disallow: *?id*_facet*_facet*&st=* for Google-Extended
- Removed Disallow: */site/searchpage.jsp?st=* for Googlebot
- Removed Disallow: *?cp=*&id=*&st=* for Googlebot
- Removed Disallow: *?id*_facet*&st=* for Googlebot
- Removed Disallow: *?id*_facet*_facet*&st=* for Googlebot
- 🤖 AIRemoved Disallow: *?cp=*&id=*&st=* for GPTBot
- 🤖 AIRemoved Disallow: *?id*_facet*&st=* for GPTBot
- 🤖 AIRemoved Disallow: *?id*_facet*_facet*&st=* for GPTBot
- 🤖 AIRemoved Disallow: *?cp=*&id=*&st=* for OAI-SearchBot
- 🤖 AIRemoved Disallow: *?id*_facet*&st=* for OAI-SearchBot
- 🤖 AIRemoved Disallow: *?id*_facet*_facet*&st=* for OAI-SearchBot
- 🤖 AIRemoved Disallow: *?cp=*&id=*&st=* for Perplexity-User
- 🤖 AIRemoved Disallow: *?id*_facet*&st=* for Perplexity-User
- 🤖 AIRemoved Disallow: *?id*_facet*_facet*&st=* for Perplexity-User
- 🤖 AIRemoved Disallow: *?cp=*&id=*&st=* for PerplexityBot
- ... and 6 more change(s)
Diff old (2026-09-03T16:37:09Z) → new (2026-09-17T15:11:14Z)
| @@ -165,11 +165,6 @@ | ||
| 165 | 165 | Disallow: *cp=8* |
| 166 | 166 | Disallow: *cp=9* |
| 167 | 167 | Disallow: *?*page=*&page=*$ |
| 168 | - | |
| 169 | - # Search Parameters Controls | |
| 170 | - Disallow: *?id*_facet*&st=* | |
| 171 | - Disallow: *?id*_facet*_facet*&st=* | |
| 172 | - Disallow: *?cp=*&id=*&st=* | |
| 173 | 168 | |
| 174 | 169 | # Malformed URL Traps (Universal Protection) |
| 175 | 170 | Disallow: *==$ |
| @@ -394,12 +389,6 @@ | ||
| 394 | 389 | Disallow: *cp=9* |
| 395 | 390 | Disallow: *?*page=*&page=*$ |
| 396 | 391 | |
| 397 | - # Search Parameters Controls | |
| 398 | - Disallow: */site/searchpage.jsp?st=* | |
| 399 | - Disallow: *?id*_facet*&st=* | |
| 400 | - Disallow: *?id*_facet*_facet*&st=* | |
| 401 | - Disallow: *?cp=*&id=*&st=* | |
| 402 | - | |
| 403 | 392 | # Malformed URL Traps |
| 404 | 393 | Disallow: *==$ |
| 405 | 394 | Disallow: */-? |