Bilibili / robots.txt
← back to bilibili.com · change detected 4d ago (2026-11-03T06:59:34Z) · modified +19 −0
What changed
- Added rules for user-agent ‘Discordbot’
- Added rules for user-agent ‘facebookexternalhit’
- Added rules for user-agent ‘Facebot’
- Added rules for user-agent ‘LinkedInBot’
- Added rules for user-agent ‘Slackbot’
- Added rules for user-agent ‘TelegramBot’
- Added rules for user-agent ‘Twitterbot’
- Added rules for user-agent ‘WhatsApp’
- Added Disallow: /ar/ for *
- Added Disallow: /en/ for *
- Added Disallow: /es/ for *
- Added Disallow: /id/ for *
- Added Disallow: /ja/ for *
- Added Disallow: /pt/ for *
- Added Disallow: /th/ for *
- Added Disallow: /zh-Hant/ for *
Diff old (2026-06-26T13:23:15Z) → new (2026-11-03T06:59:34Z)
| @@ -1,3 +1,22 @@ | ||
| 1 | 1 | User-agent: * |
| 2 | 2 | Disallow: /medialist/detail/ |
| 3 | 3 | Disallow: /index.html |
| 4 | + Disallow: /zh-Hant/ | |
| 5 | + Disallow: /en/ | |
| 6 | + Disallow: /ja/ | |
| 7 | + Disallow: /es/ | |
| 8 | + Disallow: /pt/ | |
| 9 | + Disallow: /id/ | |
| 10 | + Disallow: /th/ | |
| 11 | + Disallow: /ar/ | |
| 12 | + | |
| 13 | + User-agent: facebookexternalhit | |
| 14 | + User-agent: Facebot | |
| 15 | + User-agent: Twitterbot | |
| 16 | + User-agent: LinkedInBot | |
| 17 | + User-agent: Slackbot | |
| 18 | + User-agent: TelegramBot | |
| 19 | + User-agent: Discordbot | |
| 20 | + User-agent: WhatsApp | |
| 21 | + Disallow: /medialist/detail/ | |
| 22 | + Disallow: /index.html |