W.W. Grainger / robots.txt
← back to grainger.com · change detected 9d ago (2026-06-17T02:02:49Z) · modified +188 −167
What changed
- Response is a valid robots.txt again (was an HTML page)
- Added rules for user-agent ‘*’
- Blocked user-agent ‘AhrefsBot’ entirely
- 🤖 AIBlocked Amazonbot (Amazon) entirely
- Blocked user-agent ‘Baiduspider’ entirely
- Blocked user-agent ‘Baiduspider-render’ entirely
- Added rules for user-agent ‘bingbot’
- 🤖 AIBlocked Bytespider (ByteDance) entirely
- 🤖 AIBlocked ClaudeBot (Anthropic) entirely
- 🤖 AIBlocked FacebookBot (Meta) entirely
- Blocked user-agent ‘Google-CloudVertexBot’ entirely
- 🤖 AIBlocked Google-Extended (Google) entirely
- Blocked user-agent ‘GoogleOther’ entirely
- 🤖 AIBlocked GPTBot (OpenAI) entirely
- Blocked user-agent ‘Meta-External-Bot’ entirely
- 🤖 AIBlocked meta-externalagent (Meta) entirely
- Blocked user-agent ‘PetalBot’ entirely
- Blocked user-agent ‘voltron’ entirely
- Added sitemap https://www.grainger.com/attribute-filtered-fasteners-raw-materials-sitemap.xml
- Added sitemap https://www.grainger.com/branch-location-sitemap.xml
- Added sitemap https://www.grainger.com/brand-filtered-plumbing-lighting-sitemap.xml
- Added sitemap https://www.grainger.com/content-article-sitemap.xml
- Added sitemap https://www.grainger.com/single-filter-attribute-category-sitemap.xml
- Added sitemap https://www.grainger.com/single-filter-brand-category-sitemap.xml
- Added sitemap https://www.grainger.com/sitemap_index.xml
Diff (no previous version) → (file removed)
No line-level differences (identical or both empty).