Trend Micro / robots.txt

← back to trendmicro.com · change detected 27d ago (2026-07-08T14:47:09Z) · modified +3 −10

What changed

Removed all rules for user-agent ‘ChatGPT-User/2.0’
🤖 AIRemoved Allow: *.css$ for Amazonbot
🤖 AIRemoved Allow: *.js$ for Amazonbot
🤖 AIRemoved Disallow: /*/business/parsing-pages/ for Amazonbot
🤖 AIRemoved Disallow: /*/business/trials_parsing_pages/ for Amazonbot
🤖 AIRemoved Disallow: /download/* for Amazonbot
🤖 AIRemoved Disallow: /form/ for Amazonbot
🤖 AIRemoved Disallow: /go/ for Amazonbot
🤖 AIRemoved Disallow: /tools/* for Amazonbot
Removed Allow: *.css$ for Applebot
Removed Allow: *.js$ for Applebot
Removed Disallow: /*/business/parsing-pages/ for Applebot
Removed Disallow: /*/business/trials_parsing_pages/ for Applebot
Removed Disallow: /download/* for Applebot
Removed Disallow: /form/ for Applebot
Removed Disallow: /go/ for Applebot
Removed Disallow: /tools/* for Applebot
Removed Allow: *.css$ for Bingbot
Removed Allow: *.js$ for Bingbot
Removed Disallow: /*/business/parsing-pages/ for Bingbot
Removed Disallow: /*/business/trials_parsing_pages/ for Bingbot
Removed Disallow: /download/* for Bingbot
Removed Disallow: /form/ for Bingbot
Removed Disallow: /go/ for Bingbot
Removed Disallow: /tools/* for Bingbot
🤖 AIRemoved Allow: *.css$ for Bytespider
🤖 AIRemoved Allow: *.js$ for Bytespider
🤖 AIRemoved Disallow: /*/business/parsing-pages/ for Bytespider
🤖 AIRemoved Disallow: /*/business/trials_parsing_pages/ for Bytespider
🤖 AIRemoved Disallow: /download/* for Bytespider
🤖 AIRemoved Disallow: /form/ for Bytespider
🤖 AIRemoved Disallow: /go/ for Bytespider
🤖 AIRemoved Disallow: /tools/* for Bytespider
🤖 AIRemoved Allow: *.css$ for CCBot
🤖 AIRemoved Allow: *.js$ for CCBot
🤖 AIRemoved Disallow: /*/business/parsing-pages/ for CCBot
🤖 AIRemoved Disallow: /*/business/trials_parsing_pages/ for CCBot
🤖 AIRemoved Disallow: /download/* for CCBot
🤖 AIRemoved Disallow: /form/ for CCBot
🤖 AIRemoved Disallow: /go/ for CCBot
🤖 AIRemoved Disallow: /tools/* for CCBot
🤖 AIRemoved Allow: *.css$ for ChatGPT-User
🤖 AIRemoved Allow: *.js$ for ChatGPT-User
🤖 AIRemoved Disallow: /*/business/parsing-pages/ for ChatGPT-User
🤖 AIRemoved Disallow: /*/business/trials_parsing_pages/ for ChatGPT-User
🤖 AIRemoved Disallow: /download/* for ChatGPT-User
🤖 AIRemoved Disallow: /form/ for ChatGPT-User
🤖 AIRemoved Disallow: /go/ for ChatGPT-User
🤖 AIRemoved Disallow: /tools/* for ChatGPT-User
🤖 AIRemoved Allow: *.css$ for Claude-SearchBot
... and 111 more change(s)

Diff old (2026-06-20T14:27:25Z) → new (2026-07-08T14:47:09Z)

		@@ -1,4 +1,4 @@
1		- # Rules for Adsbot-Google (specific bot)
	1	+ # Rules for Adsbot-Google
2	2	User-agent: Adsbot-Google
3	3	Allow: /core/docs/datasheets/*.pdf$
4	4	Allow: *.js$
		@@ -35,7 +35,6 @@
35	35	# Rules for LLM/AI crawlers
36	36	User-agent: OAI-SearchBot
37	37	User-agent: ChatGPT-User
38		- User-agent: ChatGPT-User/2.0
39	38	User-agent: ClaudeBot
40	39	User-agent: claude-web
41	40	User-agent: Claude-SearchBot
		@@ -54,28 +53,22 @@
54	53	User-agent: DuckAssistBot
55	54	User-agent: cohere-ai
56	55	User-agent: CCBot
	56	+ Content-Signal: search=yes, ai-input=yes, ai-train=no
57	57	Allow: *.pdf$
58		- Allow: *.js$
59		- Allow: *.css$
60	58	Disallow: /*/config/
61	59	Disallow: /*/common/
62	60	Disallow: /*/error-messages/
63		- Disallow: /*/business/trials_parsing_pages/
64		- Disallow: /*/business/parsing-pages/
65		- Disallow: /download/*
66		- Disallow: /form/
67	61	Disallow: /ftp/
68		- Disallow: /go/
69	62	Disallow: /housecall/
70	63	Disallow: /housecall/us/
71	64	Disallow: /housecall7/
72	65	Disallow: /ja_jp/forHome/xsp/
73	66	Disallow: /machform/
74		- Disallow: /tools/*
75	67	Disallow: /?bvstate=
76	68
77	69	# Rules for all other bots
78	70	User-agent: *
	71	+ Content-Signal: search=yes, ai-input=yes, ai-train=no
79	72	Allow: /core/docs/datasheets/*.pdf$
80	73	Disallow: /*.pdf$
81	74	Disallow: /*/config/