Indian Institute of Technology Bombay / robots.txt
← back to iitb.ac.in · change detected 1mo ago (2026-09-10T18:52:44Z) · created +13 −1
What changed
- Added Disallow: /*/media/oembed for *
- Added Disallow: /README.md for *
- Added Disallow: /composer/Metapackage/README.txt for *
- Added Disallow: /composer/Plugin/ProjectMessage/README.md for *
- Added Disallow: /composer/Plugin/Scaffold/README.md for *
- Added Disallow: /composer/Plugin/VendorHardening/README.txt for *
- Added Disallow: /composer/Template/README.txt for *
- Added Disallow: /index.php/*/media/oembed for *
- Added Disallow: /index.php/media/oembed for *
- Added Disallow: /media/oembed for *
- Added Disallow: /modules/README.txt for *
- Added Disallow: /sites/README.txt for *
- Added Disallow: /themes/README.txt for *
- Removed Disallow: /README.txt for *
Diff old (2026-06-26T14:15:22Z) → new (2026-09-10T18:52:44Z)
| @@ -37,7 +37,15 @@ | ||
| 37 | 37 | Disallow: /core/ |
| 38 | 38 | Disallow: /profiles/ |
| 39 | 39 | # Files |
| 40 | - Disallow: /README.txt | |
| 40 | + Disallow: /README.md | |
| 41 | + Disallow: /composer/Metapackage/README.txt | |
| 42 | + Disallow: /composer/Plugin/ProjectMessage/README.md | |
| 43 | + Disallow: /composer/Plugin/Scaffold/README.md | |
| 44 | + Disallow: /composer/Plugin/VendorHardening/README.txt | |
| 45 | + Disallow: /composer/Template/README.txt | |
| 46 | + Disallow: /modules/README.txt | |
| 47 | + Disallow: /sites/README.txt | |
| 48 | + Disallow: /themes/README.txt | |
| 41 | 49 | Disallow: /web.config |
| 42 | 50 | # Paths (clean URLs) |
| 43 | 51 | Disallow: /admin/ |
| @@ -49,6 +57,8 @@ | ||
| 49 | 57 | Disallow: /user/password |
| 50 | 58 | Disallow: /user/login |
| 51 | 59 | Disallow: /user/logout |
| 60 | + Disallow: /media/oembed | |
| 61 | + Disallow: /*/media/oembed | |
| 52 | 62 | # Paths (no clean URLs) |
| 53 | 63 | Disallow: /index.php/admin/ |
| 54 | 64 | Disallow: /index.php/comment/reply/ |
| @@ -59,3 +69,5 @@ | ||
| 59 | 69 | Disallow: /index.php/user/register |
| 60 | 70 | Disallow: /index.php/user/login |
| 61 | 71 | Disallow: /index.php/user/logout |
| 72 | + Disallow: /index.php/media/oembed | |
| 73 | + Disallow: /index.php/*/media/oembed |