University of Cape Town / robots.txt
← back to uct.ac.za · change detected 1mo ago (2026-06-29T11:44:49Z) · modified +2 −2
What changed
- Unblocked user-agent ‘Bingbot’
- Added Disallow: (empty) for Bingbot
- 🤖 AIUnblocked OAI-SearchBot (OpenAI)
- 🤖 AIAdded Disallow: (empty) for OAI-SearchBot
Diff old (2026-06-26T14:15:22Z) → new (2026-06-29T11:44:49Z)
| @@ -72,7 +72,7 @@ | ||
| 72 | 72 | Disallow: /*?*search= |
| 73 | 73 | |
| 74 | 74 | User-agent: Bingbot |
| 75 | - Disallow: / | |
| 75 | + Disallow: | |
| 76 | 76 | |
| 77 | 77 | User-agent: Slurp |
| 78 | 78 | Disallow: /private/ |
| @@ -194,7 +194,7 @@ | ||
| 194 | 194 | Disallow: / |
| 195 | 195 | |
| 196 | 196 | User-agent: OAI-SearchBot |
| 197 | - Disallow: / | |
| 197 | + Disallow: | |
| 198 | 198 | |
| 199 | 199 | # sitemap.xml |
| 200 | 200 | Sitemap: https://uct.ac.za/sitemap.xml |