Stanford University / robots.txt snapshot
← back to stanford.edu · fetched 2026-06-26T14:15:22Z (4h ago) · HTTP 200 · 197 bytes · sha256 aadf777d47272000 · raw
final URL: https://www.stanford.edu/robots.txt
| 1 | User-Agent: * |
| 2 | Allow: / |
| 3 | Disallow: /global-components/ |
| 4 | Disallow: /test/ |
| 5 | Disallow: /editor/ |
| 6 | Disallow: /search |
| 7 | Disallow: /storybook/ |
| 8 | Disallow: /examples/ |
| 9 | |
| 10 | Sitemap: https://www.stanford.edu/sitemap.xml |