NeuralCrawl

Scale AI / robots.txt snapshot

← back to scale.com · fetched 2026-07-30T23:00:57Z (6d ago) · HTTP 200 · 1437 bytes · sha256 6faff85619c1faa3 · raw

final URL: https://scale.com/robots.txt

1# *
2User-agent: *
3Allow: /
4Allow: /api/og
5Disallow: /studio
6Disallow: /studio/*
7Disallow: /draft
8Disallow: /cdn-cgi/l/email-protection
9Disallow: /feedback/
10Disallow: /*?*utm_source=*
11Disallow: /*?*_hstc=*
12Disallow: /*?*
13Disallow: /retail/solution-brief/
14Disallow: /_makeswift/
15Disallow: /404
16Disallow: /corp/
17Disallow: /corp-api/
18Disallow: /internal/
19Disallow: /scaleai/
20Disallow: /&spoof=*
21Disallow: /events/transformx-2021/linkedin-auth
22Disallow: /events/transformx-2021/ticket-image
23Disallow: /events/transformx-2021/ticket/*
24Disallow: /events/transformx/linkedin-auth
25Disallow: /3d-sensor-fusion/*
26Disallow: /careers/legal
27Disallow: /careers/legal/*
28Disallow: /claims-intelligence
29Disallow: /content-understanding
30Disallow: /data-labeling
31Disallow: /styleguide
32Disallow: /federal-video-frame
33Disallow: /animations
34Disallow: /research
35Disallow: /research/*
36
37# Host
38Host: https://scale.com
39
40# Sitemaps
41Sitemap: https://scale.com/sitemap.xml
42Sitemap: https://scale.com/blog/sitemap.xml
43Sitemap: https://scale.com/blog/category/sitemap.xml
44Sitemap: https://scale.com/blog/author/sitemap.xml
45Sitemap: https://scale.com/customers/sitemap.xml
46Sitemap: https://scale.com/guides/sitemap.xml
47Sitemap: https://scale.com/legal/sitemap.xml
48Sitemap: https://scale.com/events/sitemap.xml
49Sitemap: https://scale.com/enterprise/prebuilt-applications/sitemap.xml
50Sitemap: https://scale.com/docs/sitemap.xml
51Sitemap: https://labs.scale.com/sitemap.xml