NeuralCrawl

๐Ÿ‡ฌ๐Ÿ‡ง Rolls-Royce

rolls-royce.com · European companies · rank #98 · Aerospace & Defense · live robots.txt ↗

AI crawler access (latest snapshot, 13h ago)

blocked restricted allowed faded = inherited from the * wildcard group

GPTBot
ChatGPT-User
OAI-SearchBot
ClaudeBot
Claude-User
Claude-SearchBot
anthropic-ai
Claude-Web
CCBot
Google-Extended
Applebot-Extended
PerplexityBot
Perplexity-User
Bytespider
Amazonbot
FacebookBot
meta-externalagent
meta-externalfetcher
cohere-ai
AI2Bot
Diffbot
omgili
YouBot
DuckAssistBot
MistralAI-User
PanguBot
Timpibot

Current robots.txt 3413 bytes · sha256 2335742e3287 · raw

User-agent: Insieve+Bot
Disallow: /
User-agent: BLP_bbot/0.1
Disallow: /
User-agent: blp_bbot/0.1
Disallow: /
User-agent: Java/1.6.0
Disallow: /
User-agent: Java/1.6.0_04
Disallow: /
User-agent: Java/1.6.0_11
Disallow: /
User-agent: Java/1.6.0_15
Disallow: /
User-agent: Java/1.6.0_16
Disallow: /
User-agent: Java/1.6.0_17
Disallow: /
User-agent: Java/1.6.0_18
Disallow: /
User-agent: Java/1.6.0_20
Disallow: /
User-agent: Java/1.6.0_22
Disallow: /
User-agent: Java/1.6.0_24
Disallow: /
User-agent: Java/1.6.0_25
Disallow: /


User-agent: *
Disallow: /*.axd$
Disallow: /~/media/Files/R/Rolls-Royce/documents/customers/marine/*
Disallow: /~/media/Files/R/Rolls-Royce/documents/marine-product-finder/*
Disallow: /staticresources/
Disallow: /investis/
Disallow: /tools/
Disallow: /investis modules/
Disallow: /*/sitecore/
Disallow: /sitecore/
Disallow: /*/sitecore/content/
Disallow: /sitecore/content/
Disallow: /favicon.ico
Disallow: /error-page.aspx
Disallow: /*/error-page.aspx
Disallow: /testcache
Disallow: /testcache.aspx
Disallow: /select-country/*
Disallow: /country-sites/new-country/
Disallow: /products-and-services/civil-aerospace/used-fleet-flyer/*
Disallow: /media/media-contacts
Disallow: /your-virtual-resources
Disallow: /usa-your-virtual-resources
Disallow: /apac-your-virtual-resources
Disallow: /media/our-stories/innovation/2018/introducing-accel
Disallow: /media/our-stories/innovation/2018/introducing-accel/*
Disallow: /media/our-stories/innovation/2018/introducing-accel.aspx
Disallow: /customers/civil-aerospace/products/civil-large-engines/integrated_marketing_communications.aspx
Disallow: /~/media/Files/R/Rolls-Royce/documents/investors/c-share-historical-information-jan-2019-v1.pdf
Disallow: /~/media/Files/R/Rolls-Royce/documents/investors/historical-c-share-information-january-2009-to-january-2020.pdf
Disallow: /~/media/Files/R/Rolls-Royce/documents/Rolls_Royce_Presentation_Thought_Leadership_Richard_Partridge_Conrad_Banks.pdf
Disallow: /about/our-story/the-rolls-royce-heritage-trust/branches/bristol-branch.aspx#collections-and-exhibits
Disallow: /about/our-story/the-rolls-royce-heritage-trust/branches/bristol-branch
Disallow: /about/our-story/the-rolls-royce-heritage-trust/branches/bristol-branch.aspx
Disallow: /about/our-story/the-rolls-royce-heritage-trust/branches/bristol-branch#collections-and-exhibits
Disallow: /about/our-story/the-rolls-royce-heritage-trust/branches/bristol-branch.aspx#collections-and-exhibits.aspx
Disallow: /products-and-services/civil-aerospace/widebody/trent-1000/its-time-for-the-trent-1000.aspx
Disallow: /~/media/Files/R/Rolls-Royce/documents/defence/vcomb-3340-naval-portfolio.pdf
Disallow: /~/media/Files/R/Rolls-Royce/documents/defence/VCOMB_3340_Naval_Portfolio_2022.pdf
Disallow: /~/media/Files/R/Rolls-Royce/documents/defence/VCOMB_3340_Naval_Portfolio_2022_v2.pdf
Disallow: /~/media/Files/R/Rolls-Royce/documents/defence/VCOMB_3340_Naval_Portfolio_Korean_2022.pdf
Disallow: /~/media/Files/R/Rolls-Royce/documents/sustainability/Supplier Code of Conduct_AUG2018_EN.pdf
Disallow: /~/media/Files/R/Rolls-Royce/documents/sustainability/supplier-global-code-tcm92-51635.pdf
Disallow: /~/media/Files/R/Rolls-Royce/documents/careers/impotant-info-help-and-tips-for-online-assessments.pdf
Disallow: /site-services/on-wing-services-tool-bailment-request-form/tooling-bailment-inquiry.aspx

Sitemap: https://www.rolls-royce.com/sitemap.xml

Change history

  1. initial snapshot
    • First snapshot of robots.txt archived