NeuralCrawl

Ben-Gurion University of the Negev / robots.txt snapshot

← back to bgu.ac.il · fetched 2026-06-26T23:05:56Z (4h ago) · HTTP 200 · 158 bytes · sha256 2b182b5f7750c6b3 · raw

final URL: https://www.bgu.ac.il/robots.txt

1User-agent: *
2Allow: /
3
4Disallow: *print
5Disallow: */favorites
6Disallow: */search-result
7Disallow: */combinations
8
9Sitemap: https://www.bgu.ac.il/sitemapxml/