NeuralCrawl

University of Tokyo / robots.txt snapshot

← back to u-tokyo.ac.jp · fetched 2026-06-26T14:15:22Z (5h ago) · HTTP 200 · 124 bytes · sha256 d5c34f6ec881861e · raw

final URL: https://www.u-tokyo.ac.jp/robots.txt

1User-agent: *
2Disallow: /gakunai/
3#
4# User-agent: Twitterbot
5# Allow: /images/
6#
7# Sitemap: https://www.u-tokyo.ac.jp/XXX