NeuralCrawl

๐Ÿ‡ฉ๐Ÿ‡ช SAP

sap.com · European companies · rank #28 · Technology · live robots.txt ↗

AI crawler access (latest snapshot, 13h ago)

blocked restricted allowed faded = inherited from the * wildcard group

GPTBot
ChatGPT-User
OAI-SearchBot
ClaudeBot
Claude-User
Claude-SearchBot
anthropic-ai
Claude-Web
CCBot
Google-Extended
Applebot-Extended
PerplexityBot
Perplexity-User
Bytespider
Amazonbot
FacebookBot
meta-externalagent
meta-externalfetcher
cohere-ai
AI2Bot
Diffbot
omgili
YouBot
DuckAssistBot
MistralAI-User
PanguBot
Timpibot

Current robots.txt 3006 bytes · sha256 3a41b4d1c48a · raw

#
# Welcome to SAP SE 
# ____    _    ____  
#/ ___|  / \  |  _ \ 
#\___ \ / _ \ | |_) |
# ___) / ___ \|  __/ 
#|____/_/   \_\_|  
#
# robots.txt for https://www.sap.com
#
# https://www.sap.com/about.html
#
# Find your next job: https://www.sap.com/careers
#
# Last updated 2026-05-20

User-agent: *
Disallow: /

# Search Engines & AI Agents (all share the same rule set)
User-agent: Googlebot
User-agent: Googlebot-Image
User-agent: Googlebot-News
User-agent: Googlebot-Video
User-agent: GoogleOther
User-agent: Google-InspectionTool
User-agent: Storebot-Google
User-agent: Bingbot
User-agent: 360Spider
User-agent: Baiduspider
User-agent: coccocbot
User-agent: Daum
User-agent: DuckDuckBot
User-agent: Ecosia_bot
User-agent: MojeekBot
User-agent: Yeti
User-agent: SeznamBot
User-agent: Sogou web spider
User-agent: Yahoo! Slurp
User-agent: YandexAccessibilityBot
User-agent: YandexMobileBot
User-agent: Yandex
User-agent: Amazonbot
User-agent: ClaudeBot
User-agent: Claude-User
User-agent: Claude-SearchBot
User-agent: CCBot
User-agent: Google-Extended
User-agent: FacebookBot
User-agent: MistralAI-User
User-agent: GPTBot
User-agent: ChatGPT-User
User-agent: OAI-SearchBot
User-agent: PerplexityBot
User-agent: Perplexity-User
User-agent: archive.org_bot
Disallow: /bin/sapdx/
Disallow: /bin/sapdxc/
Disallow: /design-system/digital/
Disallow: /dmc/
Disallow: /heartbeat.html
Disallow: /investors/de/stock/fixed-income/bond-documentation*
Disallow: /investors/de/stock/fixed-income/cp-documentation* 
Disallow: /investors/en/stock/fixed-income/bond-documentation*
Disallow: /investors/en/stock/fixed-income/cp-documentation* 
Disallow: */_jcr_content*.html
Disallow: */abm/*
Disallow: */cmp/dg/gdpr/*
Disallow: */cmp/dg/gdpr-emea/*
Disallow: */cmp/dg/gdpr-germany/*
Disallow: */cmp/eml/*
Disallow: */cmp/oth/crm-wx18-xrg-geb-fba/*
Disallow: */cmp/oth/crm-wx18-xrg-geb-supfba/*
Disallow: */cmp/oth/gdpr-opt-out/*
Disallow: */cmp/spm/*
Disallow: */cmp/td/sap-sustainability-data-exchange*
Disallow: */events/sapphire/orlando/celebration-night-concert.html
Disallow: */events/sponsorships.html
Disallow: */legal/video-surveillance*
Disallow: */products/erp/s4hana-private-edition/trial.html
Disallow: */tye.html
Disallow: */typ.html
Disallow: */registration/*?
Allow: /abm/dod.html
Allow: /abm/dod/*
Allow: /dmc/exp/*

# Fetcher and other
User-agent: AdsBot-Google
User-agent: AdsBot-Google-Mobile
User-agent: Chrome-Lighthouse
User-agent: facebookexternalhit
User-agent: Google-Read-Aloud
User-agent: Google-Safety
User-agent: Hatena
User-agent: linkedinbot
User-agent: Mediapartners-Google
User-agent: Pinterestbot
User-agent: SchemaBot
User-agent: SiteAuditBot
User-agent: SiteimproveBot
User-agent: Slackbot-LinkExpanding
User-agent: Telegram
User-agent: Twitterbot
Allow: /

Sitemap: https://www.sap.com/sitemap_index.xml
Sitemap: https://www.sap.com/sitemap-index.xml  
Sitemap: https://www.sap.com/sitemap_siteimprove_index.xml
Sitemap: https://www.sap.com/sitemap_us_int_only_siteimprove_index.xml

Change history

  1. initial snapshot
    • First snapshot of robots.txt archived