NeuralCrawl

Union Pacific / robots.txt snapshot

← back to up.com · fetched 2026-06-20T01:10:30Z (18h ago) · HTTP 200 · 927 bytes · sha256 ff2c5dc48cb5ebf4 · raw

final URL: https://www.up.com/robots.txt

1# robots.txt for https://www.up.com/
2
3User-agent: *
4
5# Disallow internal AEM system paths
6Disallow: /libs/
7Disallow: /apps/
8Disallow: /conf/
9Disallow: /etc/
10Disallow: /bin/
11Disallow: /var/
12Disallow: /system/
13Disallow: /home/users/
14
15# Disallow DAM subpaths not intended for indexing
16Disallow: /content/dam/.*/renditions/
17Disallow: /content/dam/.*/jcr:content/
18
19# Disallow non-public or template content
20Disallow: /content/experience-fragments/
21
22# Disallow internal search and contact utilities
23Disallow: /search
24Disallow: /search/
25Disallow: /search.html
26Disallow: /content/up/en/search
27Disallow: /content/up/en/search.html
28Disallow: /messages/index.cfm
29Disallow: /about-us/privacy/inquiry
30Disallow: /about-us/privacy/inquiry/
31Disallow: /content/upcom/us/en/about-us/privacy/inquiry.html
32
33
34# Allow key public-facing content
35Allow: /content/up/en/
36Allow: /content/dam/
37
38# Sitemap reference
39Sitemap: https://www.up.com/sitemap.xml