Governments
The archive — every monitored site, its robots.txt history and AI-crawler stance.
All sites
Top 1000 websites
US 500
Nasdaq 100
AI models
Cybersecurity
SEO & AI search
Creative rights-holders
Knowledge & UGC
Academic & open research
Universities
National indices
Banks & fintech
E-commerce
European companies
Social Networks
Governments
Publishers
| # | Company | Domain | Sector | robots.txt | AI bots blocked | Snapshots | Changes | Last change | |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 🇺🇸 | United States (USA.gov) | usa.gov | Americas | present | 0 | 1 | 0 | — |
| 2 | 🇨🇦 | Canada (Government of Canada) | canada.ca | Americas | present | 0 | 1 | 0 | — |
| 3 | 🇬🇧 | United Kingdom (GOV.UK) | gov.uk | Europe | present | 0 | 1 | 0 | — |
| 4 | 🇫🇷 | France (Gouvernement) | gouvernement.fr | Europe | present | 0 | 2 | 1 | 1mo ago |
| 5 | 🇩🇪 | Germany (Bundesregierung) | bundesregierung.de | Europe | present | 0 | 1 | 0 | — |
| 6 | 🇮🇹 | Italy (Governo) | governo.it | Europe | present | 0 | 1 | 0 | — |
| 7 | 🇪🇸 | Spain (La Moncloa) | lamoncloa.gob.es | Europe | present | 0 | 1 | 0 | — |
| 8 | 🇳🇱 | Netherlands (Government.nl) | government.nl | Europe | present | 0 | 1 | 0 | — |
| 9 | 🇧🇪 | Belgium (Belgium.be) | belgium.be | Europe | present | 0 | 1 | 0 | — |
| 10 | 🇮🇪 | Ireland (Gov.ie) | gov.ie | Europe | present | 🤖 6 | 3 | 2 | 3mo ago |
| 11 | 🇵🇹 | Portugal (Governo) | portugal.gov.pt | Europe | present | 0 | 1 | 0 | — |
| 12 | 🇸🇪 | Sweden (Regeringen) | government.se | Europe | present | 0 | 3 | 2 | 1mo ago |
| 13 | 🇳🇴 | Norway (Regjeringen) | regjeringen.no | Europe | present | 0 | 1 | 0 | — |
| 14 | 🇩🇰 | Denmark (Statsministeriet) | stm.dk | Europe | present | 0 | 1 | 0 | — |
| 15 | 🇫🇮 | Finland (Valtioneuvosto) | valtioneuvosto.fi | Europe | present | 0 | 1 | 0 | — |
| 16 | 🇮🇸 | Iceland (Government.is) | government.is | Europe | present | 🤖 13 | 9 | 8 | 18d ago |
| 17 | 🇨🇭 | Switzerland (Admin.ch) | admin.ch | Europe | present | 0 | 1 | 0 | — |
| 18 | 🇦🇹 | Austria (Bundeskanzleramt) | bundeskanzleramt.gv.at | Europe | present | 0 | 1 | 0 | — |
| 19 | 🇱🇺 | Luxembourg (Gouvernement) | gouvernement.lu | Europe | present | 0 | 2 | 1 | 4mo ago |
| 20 | 🇵🇱 | Poland (Gov.pl) | gov.pl | Europe | present | 0 | 1 | 0 | — |
| 21 | 🇨🇿 | Czech Republic (Vlada) | vlada.gov.cz | Europe | present | 0 | 1 | 0 | — |
| 22 | 🇸🇰 | Slovakia (Vlada) | vlada.gov.sk | Europe | absent | 0 | 0 | 0 | — |
| 23 | 🇸🇮 | Slovenia (Gov.si) | gov.si | Europe | present | 0 | 1 | 0 | — |
| 24 | 🇭🇷 | Croatia (Vlada) | vlada.gov.hr | Europe | absent | 0 | 0 | 0 | — |
| 26 | 🇧🇬 | Bulgaria (Government) | government.bg | Europe | present | 0 | 1 | 0 | — |
| 27 | 🇬🇷 | Greece (Government) | government.gov.gr | Europe | present | 0 | 1 | 0 | — |
| 29 | 🇨🇾 | Cyprus (Gov.cy) | gov.cy | Europe | present | 0 | 1 | 0 | — |
| 31 | 🇱🇻 | Latvia (Ministru kabinets) | mk.gov.lv | Europe | present | 0 | 1 | 0 | — |
| 32 | 🇱🇹 | Lithuania (LRV) | lrv.lt | Europe | present | 0 | 1 | 0 | — |
| 33 | 🇭🇺 | Hungary (Kormany) | kormany.hu | Europe | present | 0 | 1 | 0 | — |
| 34 | 🇦🇱 | Albania (Kryeministria) | kryeministria.al | Europe | present | 0 | 1 | 0 | — |
| 35 | 🇲🇰 | North Macedonia (Vlada) | vlada.mk | Europe | present | 0 | 1 | 0 | — |
| 36 | 🇲🇪 | Montenegro (Gov.me) | gov.me | Europe | present | 0 | 1 | 0 | — |
| 37 | 🇲🇩 | Moldova (Gov.md) | gov.md | Europe | present | 0 | 1 | 0 | — |
| 38 | 🇺🇦 | Ukraine (KMU) | kmu.gov.ua | Europe | present | 0 | 9 | 8 | 3mo ago |
| 39 | 🇦🇲 | Armenia (Gov.am) | gov.am | Europe | present | 0 | 2 | 1 | 1mo ago |
| 40 | 🇯🇵 | Japan (Kantei) | japan.go.jp | Asia-Pacific | absent | 0 | 0 | 0 | — |
| 41 | 🇰🇷 | South Korea (Korea.kr) | korea.kr | Asia-Pacific | present | 0 | 1 | 0 | — |
| 43 | 🇮🇳 | India (India.gov.in) | india.gov.in | Asia-Pacific | present | 0 | 1 | 0 | — |
| 44 | 🇦🇺 | Australia (my.gov.au) | my.gov.au | Asia-Pacific | present | 0 | 1 | 0 | — |
| 45 | 🇳🇿 | New Zealand (Govt.nz) | govt.nz | Asia-Pacific | absent | 0 | 0 | 0 | — |
| 46 | 🇮🇩 | Indonesia (Indonesia.go.id) | indonesia.go.id | Asia-Pacific | absent | 0 | 0 | 0 | — |
| 47 | 🇲🇾 | Malaysia (Malaysia.gov.my) | malaysia.gov.my | Asia-Pacific | present | 0 | 1 | 0 | — |
| 52 | 🇧🇼 | Botswana (Gov.bw) | gov.bw | Africa | present | 0 | 1 | 0 | — |
| 53 | 🇬🇭 | Ghana (Ghana.gov.gh) | ghana.gov.gh | Africa | present | 0 | 1 | 0 | — |
| 55 | 🇲🇺 | Mauritius (Govmu) | govmu.org | Africa | absent | 0 | 0 | 0 | — |
| 56 | 🇨🇻 | Cape Verde (Governo) | governo.cv | Africa | present | 0 | 1 | 0 | — |
| 57 | 🇧🇷 | Brazil (Gov.br) | gov.br | Americas | present | 0 | 1 | 0 | — |
| 58 | 🇦🇷 | Argentina (Argentina.gob.ar) | argentina.gob.ar | Americas | present | 0 | 2 | 1 | 1mo ago |
| 60 | 🇺🇾 | Uruguay (Gub.uy) | gub.uy | Americas | present | 0 | 1 | 0 | — |
| 61 | 🇨🇷 | Costa Rica (Presidencia) | presidencia.go.cr | Americas | present | 🤖 13 | 5 | 4 | 15d ago |
| 62 | 🇲🇽 | Mexico (Gob.mx) | gob.mx | Americas | present | 0 | 1 | 0 | — |
| 63 | 🇨🇴 | Colombia (Presidencia) | presidencia.gov.co | Americas | present | 0 | 7 | 11 | 2mo ago |
| 64 | 🇵🇪 | Peru (Gob.pe) | gob.pe | Americas | present | 0 | 2 | 1 | 2mo ago |
| 65 | 🇵🇦 | Panama (Presidencia) | presidencia.gob.pa | Americas | present | 0 | 1 | 0 | — |
| 66 | 🇩🇴 | Dominican Republic (Presidencia) | presidencia.gob.do | Americas | present | 0 | 1 | 0 | — |
| 67 | 🇯🇲 | Jamaica (Office of the PM) | opm.gov.jm | Americas | present | 0 | 1 | 0 | — |
| 68 | 🇹🇹 | Trinidad and Tobago (Gov.tt) | gov.tt | Americas | absent | 0 | 0 | 0 | — |
| 69 | 🇬🇾 | Guyana (Office of the President) | op.gov.gy | Americas | present | 🤖 13 | 6 | 5 | 13d ago |
| 70 | 🇸🇷 | Suriname (Gov.sr) | gov.sr | Americas | present | 0 | 1 | 0 | — |