NeuralCrawl

🇧🇷 University of São Paulo

usp.br · Universities · rank #40 · University · live robots.txt ↗ · redirect

robots.txt fetch status

The server answered but redirected /robots.txt elsewhere instead of serving a robots policy file.

Automated fetch: Redirect (1 redirect): https://www.usp.br/robots.txt → https://www5.usp.br (cross-host); received HTML

What the server returned

Preview of https://www5.usp.br (not a valid robots.txt).

Screenshot of the page served at robots.txt
I captured this screenshot and I will cache it for later visits :)
Raw HTML response (archived)

Change history

No archived robots.txt yet. The server answered but redirected /robots.txt elsewhere instead of serving a robots policy file.

Automated fetch: Redirect (1 redirect): https://www.usp.br/robots.txt → https://www5.usp.br (cross-host); received HTML