NeuralCrawl

🇺🇸 Advance Auto Parts

advanceautoparts.com · US 500 · rank #387 · Retailing · live robots.txt ↗

AI crawler access (latest snapshot, 5d ago)

blocked restricted allowed faded = inherited from the * wildcard group

GPTBot
ChatGPT-User
OAI-SearchBot
ClaudeBot
Claude-User
Claude-SearchBot
anthropic-ai
Claude-Web
CCBot
Google-Extended
Applebot-Extended
PerplexityBot
Perplexity-User
Bytespider
Amazonbot
FacebookBot
meta-externalagent
meta-externalfetcher
cohere-ai
AI2Bot
Diffbot
omgili
YouBot
DuckAssistBot
MistralAI-User
PanguBot
Timpibot

Current robots.txt 698 bytes · sha256 e1b22290a370 · raw

User-agent: Mediapartners-Google
Disallow:

User-agent: *
Disallow: */LogonForm*
Disallow: */ForgotPasswordView*
Disallow: */UserRegistrationForm*
Disallow: */Address*
Disallow: */TemporaryPasswordSentForm
Disallow: */*Order*
Disallow: */WriteAReviewView*
Disallow: */StoreLocatorDetailView*
Disallow: */StoreLocatorView*

#Disallow SearchResults urls
Disallow: */*?sortBy=MOST_POPULAR
Disallow: */*?sortBy=TOP_RATED
Disallow: */*?sortBy=PRICE_LOW_HIGH
Disallow: */*?sortBy=PRICE_HIGH_LOW
Disallow: */*?sortBy=NAME_A_Z
Disallow: */*?sortBy=NAME_Z_A
Disallow: */web/* 
Disallow: */*storeCode=*
Disallow: */*searchTerm=*
Disallow: */*storeId=*

Sitemap: https://shop.advanceautoparts.com/sitemap.xml

Compare any two snapshots

Change history

  1. modified +27 −0 view diff → snapshot
    • Added rules for user-agent ‘*’
    • Added rules for user-agent ‘Mediapartners-Google’
    • Added sitemap https://shop.advanceautoparts.com/sitemap.xml
  2. modified +0 −27 view diff → snapshot
    • Removed all rules for user-agent ‘*’
    • Removed all rules for user-agent ‘Mediapartners-Google’
    • Removed sitemap https://shop.advanceautoparts.com/sitemap.xml
  3. initial snapshot
    • First snapshot of robots.txt archived