IMDb / robots.txt
← back to imdb.com · change detected 2mo ago (2026-08-28T02:24:20Z) · modified +11 −1
What changed
- Removed Disallow: /*/contribute/* for AmazonAdBot
- Removed Disallow: /*/register for AmazonAdBot
- Removed Disallow: /*/registration/ for AmazonAdBot
- Removed Disallow: /*/title/*/review/* for AmazonAdBot
- Removed Disallow: /*/title/*/reviews for AmazonAdBot
- Removed Disallow: /*/user/ur*/reviews for AmazonAdBot
- Removed Disallow: /contribute/* for AmazonAdBot
- Removed Disallow: /register for AmazonAdBot
- Removed Disallow: /registration/ for AmazonAdBot
- Removed Disallow: /title/*/review/* for AmazonAdBot
- Removed Disallow: /title/*/reviews for AmazonAdBot
- Removed Disallow: /user/ur*/reviews for AmazonAdBot
Diff old (2026-08-25T01:27:49Z) → new (2026-08-28T02:24:20Z)
| @@ -35,7 +35,6 @@ | ||
| 35 | 35 | User-agent: Discordbot |
| 36 | 36 | User-agent: Snapchat |
| 37 | 37 | User-agent: Slackbot-LinkExpanding |
| 38 | - User-agent: AmazonAdBot | |
| 39 | 38 | User-agent: Amazonbot |
| 40 | 39 | User-agent: Amzn-SearchBot |
| 41 | 40 | User-agent: Amzn-User |
| @@ -63,5 +62,16 @@ | ||
| 63 | 62 | Disallow: /user/ur*/reviews |
| 64 | 63 | Disallow: /*/user/ur*/reviews |
| 65 | 64 | |
| 65 | + User-agent: AmazonAdBot | |
| 66 | + Allow: / | |
| 67 | + Disallow: /find | |
| 68 | + Disallow: /*/find | |
| 69 | + Disallow: /find/ | |
| 70 | + Disallow: /*/find/ | |
| 71 | + Disallow: /search/title/* | |
| 72 | + Disallow: /*/search/title/* | |
| 73 | + Disallow: /search/name/* | |
| 74 | + Disallow: /*/search/name/* | |
| 75 | + | |
| 66 | 76 | User-agent: * |
| 67 | 77 | Disallow: / |