NeuralCrawl

Scale AI / robots.txt snapshot

← back to scale.com · fetched 2026-06-20T01:10:30Z (18h ago) · HTTP 200 · 1422 bytes · sha256 a63d430b2deafe19 · raw

final URL: https://scale.com/robots.txt

1# *
2User-agent: *
3Allow: /
4Disallow: /studio
5Disallow: /studio/*
6Disallow: /draft
7Disallow: /cdn-cgi/l/email-protection
8Disallow: /feedback/
9Disallow: /*?*utm_source=*
10Disallow: /*?*_hstc=*
11Disallow: /*?*
12Disallow: /retail/solution-brief/
13Disallow: /_makeswift/
14Disallow: /404
15Disallow: /corp/
16Disallow: /corp-api/
17Disallow: /internal/
18Disallow: /scaleai/
19Disallow: /&spoof=*
20Disallow: /events/transformx-2021/linkedin-auth
21Disallow: /events/transformx-2021/ticket-image
22Disallow: /events/transformx-2021/ticket/*
23Disallow: /events/transformx/linkedin-auth
24Disallow: /3d-sensor-fusion/*
25Disallow: /careers/legal
26Disallow: /careers/legal/*
27Disallow: /claims-intelligence
28Disallow: /content-understanding
29Disallow: /data-labeling
30Disallow: /styleguide
31Disallow: /federal-video-frame
32Disallow: /animations
33Disallow: /research
34Disallow: /research/*
35
36# Host
37Host: https://scale.com
38
39# Sitemaps
40Sitemap: https://scale.com/sitemap.xml
41Sitemap: https://scale.com/blog/sitemap.xml
42Sitemap: https://scale.com/blog/category/sitemap.xml
43Sitemap: https://scale.com/blog/author/sitemap.xml
44Sitemap: https://scale.com/customers/sitemap.xml
45Sitemap: https://scale.com/guides/sitemap.xml
46Sitemap: https://scale.com/legal/sitemap.xml
47Sitemap: https://scale.com/events/sitemap.xml
48Sitemap: https://scale.com/enterprise/prebuilt-applications/sitemap.xml
49Sitemap: https://scale.com/docs/sitemap.xml
50Sitemap: https://labs.scale.com/sitemap.xml