Le Monde / robots.txt

← back to lemonde.fr · change detected 21d ago (2026-07-16T11:40:45Z) · modified +54 −0

What changed

		@@ -903,3 +903,57 @@
903	903
904	904	User-agent: xAI-SearchBot
905	905	Disallow: /
	906	+
	907	+ User-agent: AIWebIndex
	908	+ Disallow: /
	909	+
	910	+ User-agent: amazon-kendra
	911	+ Disallow: /
	912	+
	913	+ User-agent: amazon-QBusiness
	914	+ Disallow: /
	915	+
	916	+ User-agent: bedrockbot
	917	+ Disallow: /
	918	+
	919	+ User-agent: Crawlspace
	920	+ Disallow: /
	921	+
	922	+ User-agent: Doubaobot
	923	+ Disallow: /
	924	+
	925	+ User-agent: ERNIEBot
	926	+ Disallow: /
	927	+
	928	+ User-agent: KimiCrawler
	929	+ Disallow: /
	930	+
	931	+ User-agent: micro-crawl
	932	+ Disallow: /
	933	+
	934	+ User-agent: MoonshotBot
	935	+ Disallow: /
	936	+
	937	+ User-agent: MoonSpider
	938	+ Disallow: /
	939	+
	940	+ User-agent: Mozilla-Tabstack
	941	+ Disallow: /
	942	+
	943	+ User-agent: QwenBot
	944	+ Disallow: /
	945	+
	946	+ User-agent: SalamandraVLM
	947	+ Disallow: /
	948	+
	949	+ User-agent: TongyiBot
	950	+ Disallow: /
	951	+
	952	+ User-agent: YandexAdditional
	953	+ Disallow: /
	954	+
	955	+ User-agent: YandexAdditionalBot
	956	+ Disallow: /
	957	+
	958	+ User-agent: YiyanBot
	959	+ Disallow: /