The Asahi Shimbun / robots.txt
← back to asahi.com · change detected 17h ago (2026-08-04T06:35:30Z) · modified +1 −0
What changed
- 🤖 AIAdded Allow: /ads/ for OAI-SearchBot
Diff old (2026-06-20T01:10:30Z) → new (2026-08-04T06:35:30Z)
| @@ -21,6 +21,7 @@ | ||
| 21 | 21 | |
| 22 | 22 | User-agent: OAI-SearchBot |
| 23 | 23 | Disallow: / |
| 24 | + Allow: /ads/ | |
| 24 | 25 | |
| 25 | 26 | User-agent: ChatGPT-User |
| 26 | 27 | Disallow: / |