Ireland (Gov.ie) / robots.txt snapshot
← back to gov.ie · fetched 2026-06-24T10:04:25Z (1d ago) · HTTP 200 · 991 bytes · sha256 3f91b2d47ed913d0 · raw
final URL: https://www.gov.ie/robots.txt
| 1 | # Amazon AI |
| 2 | User-agent: Amazonbot |
| 3 | Disallow: / |
| 4 | |
| 5 | # Apple Web Crawler |
| 6 | User-agent: Applebot |
| 7 | User-agent: Applebot-Extended |
| 8 | Disallow: / |
| 9 | |
| 10 | # Baidu |
| 11 | User-agent: Baiduspider |
| 12 | User-agent: Baiduspider-video |
| 13 | User-agent: Baiduspider-image |
| 14 | User-agent: ERNIEBot |
| 15 | User-agent: YiyanBot |
| 16 | Disallow: / |
| 17 | |
| 18 | # Anthropic AI |
| 19 | User-agent: ClaudeBot |
| 20 | Disallow: / |
| 21 | |
| 22 | # DataForSEO |
| 23 | User-agent: DataForSeoBot |
| 24 | Disallow: / |
| 25 | |
| 26 | # Google AI |
| 27 | User-agent: Google-Extended |
| 28 | Disallow: / |
| 29 | |
| 30 | # OpenAI |
| 31 | User-agent: GPTBot |
| 32 | Disallow: / |
| 33 | |
| 34 | # Meta (Facebook, Instagram, WhatsApp) |
| 35 | User-agent: Meta-ExternalAgent |
| 36 | Disallow: / |
| 37 | |
| 38 | # Majestic SEO |
| 39 | User-agent: MJ12bot |
| 40 | Disallow: / |
| 41 | |
| 42 | # Petal Search |
| 43 | User-agent: PetalBot |
| 44 | Disallow: / |
| 45 | |
| 46 | # SoGou |
| 47 | User-agent: Sogou Spider |
| 48 | Disallow: / |
| 49 | |
| 50 | # Yandex |
| 51 | User-agent: Yandex |
| 52 | Disallow: / |
| 53 | |
| 54 | User-agent: * |
| 55 | Disallow: /documents/ |
| 56 | Disallow: /search/ |
| 57 | Disallow: /*/search/ |
| 58 | Disallow: /cuardaigh/ |
| 59 | Disallow: /*/cuardaigh/ |
| 60 | # Blocks numerous Irish search pages |
| 61 | Disallow: /*?* |
| 62 | Allow: /static/ |
| 63 | |
| 64 | # Point to sitemap |
| 65 | Sitemap: https://www.gov.ie/sitemap.xml |