NeuralCrawl

robots.txt Visualizer

Enter any website — we'll fetch its live robots.txt and show it with the same colour-coding and AI/search-crawler analysis used across NeuralCrawl.

Try a domain like nytimes.com, apple.com, or your own site. Fetches run server-side from this host — nothing is stored when you use the visualizer.