Hyundai Motor / robots.txt
← back to hyundai.com · change detected 1d ago (2026-11-06T08:00:37Z) · modified +5 −2
What changed
- Added rules for user-agent ‘AhrefsBot’
- Added Disallow: /it/it/ for MJ12bot
Diff old (2026-11-03T06:59:34Z) → new (2026-11-06T08:00:37Z)
| @@ -1,4 +1,3 @@ | ||
| 1 | - | |
| 2 | 1 | # ------------------------------------------------------------ |
| 3 | 2 | # 1. Priority AI/Search Crawler Access |
| 4 | 3 | # ------------------------------------------------------------ |
| @@ -102,12 +101,16 @@ | ||
| 102 | 101 | |
| 103 | 102 | User-agent: dotbot |
| 104 | 103 | User-agent: oBot |
| 105 | - User-agent: MJ12bot | |
| 104 | + User-agent: AhrefsBot | |
| 106 | 105 | Disallow: /nl/nl/ |
| 107 | 106 | |
| 108 | 107 | User-agent: CazoodleBot |
| 109 | 108 | User-agent: dotbot/1.0 |
| 110 | 109 | User-agent: Gigabot |
| 110 | + Disallow: /it/it/ | |
| 111 | + | |
| 112 | + User-agent: MJ12bot | |
| 113 | + Disallow: /nl/nl/ | |
| 111 | 114 | Disallow: /it/it/ |
| 112 | 115 | |
| 113 | 116 | User-agent: ia_archiver |