NeuralCrawl

Palo Alto Networks / robots.txt

← back to paloaltonetworks.com · change detected 28d ago (2026-07-07T02:22:00Z) · modified +108 −0

What changed

Diff old (2026-06-20T01:10:31Z) → new (2026-07-07T02:22:00Z)

@@ -110,3 +110,111 @@
110110  Allow: /*.gif$
111111  Allow: /*.png$
112112  Allow: /*.jpg$
113+
114+ User-agent: OAI-SearchBot
115+ User-agent: ChatGPT-User
116+ User-agent: PerplexityBot
117+ User-agent: Claude-SearchBot
118+ User-agent: Claude-User
119+ Allow: /sitemap.xml
120+ Allow: /apps/pan/public/downloadResource
121+ Allow: /apps/pan/public/singlePageReactModel?pageId=
122+ Allow: /apps/pan/public/unit42/bloglist.loadblogresults.json?searchLanguage=en_US
123+ Allow: /blog/wp-admin/admin-ajax.php
124+ Allow: /devsectalks/wp-admin/admin-ajax.php
125+ Allow: /perspectives/wp-admin/admin-ajax.php
126+ Allow: /content/dam/*.svg$
127+ Allow: /content/dam/*.gif$
128+ Allow: /content/dam/*.png$
129+ Allow: /content/dam/*.jpg$
130+ Allow: /content/dam/*.jpeg$
131+ Allow: /content/dam/*.pdf$
132+ Allow: /content/dam/*.gif?imwidth
133+ Allow: /content/dam/*.png?imwidth
134+ Allow: /content/dam/*.jpg?imwidth
135+ Allow: /content/dam/*.jpeg?imwidth
136+ Allow: /content/dam/*.js$
137+ Allow: /content/dam/*.css$
138+ Disallow: /content/dam/
139+ Disallow: /content/dam/pan/en_US/assets/pdf/legal/archive/
140+ Disallow: /content/pan/en_US/content/dam
141+ Disallow: /content/pan/en_US/partners/nextwave-partner-portal/
142+ Disallow: /content/pan/en_US/field/
143+ Disallow: /content/pan/en_US/ignitePromoDemo
144+ Disallow: /content/pan/en_US/legal-notices/privacy/privacy-notice
145+ Disallow: /content/pan/en_US/
146+ Disallow: /content/pan/ja_JP/
147+ Disallow: /content/pan/de_DE/
148+ Disallow: /content/pan/fr_FR/
149+ Disallow: /content/pan/es_ES/
150+ Disallow: /content/pan/pt_BR/
151+ Disallow: /content/pan/en_AU/
152+ Disallow: /content/pan/zh_TW/
153+ Disallow: /content/pan/ko_KR/
154+ Disallow: /content/pan/zh_CN/
155+ Disallow: /content/pan/es_LA/
156+ Disallow: /content/pan/en_US/allsitemap.html
157+ Disallow: /static/
158+ Disallow: /static_260619/
159+ Disallow: /static_perf/
160+ Disallow: /qa/
161+ Disallow: /allsitemap
162+ Disallow: /allsitemap.html
163+ Disallow: /content/pan/en_US/sitemaps/
164+ Disallow: /sitemaps/
165+ Disallow: /content/pan/en_GB/
166+ Disallow: /content/pan/en_CA/
167+ Disallow: /content/pan/en_SG/
168+ Disallow: /content/pan/en_IN/
169+ Disallow: /content/pan/it_IT/
170+ Disallow: /content/pan/es_MX/
171+ Disallow: /content/pan/en_US/topNav/
172+ Disallow: /content/pan/en_US/newTopNav/
173+ Disallow: /content/pan/en_US/expandedTopNav/
174+ Disallow: /apps/
175+ Disallow: /content/pan-dev
176+ Disallow: /content/pan/en_US/healthCheck.html
177+ Disallow: /content/pan/en_US/events/xdr-event
178+ Disallow: /content/pan/en_US/events/xdr-event.html
179+ Disallow: /content/pan/en_US/events/xdr-post-event
180+ Disallow: /content/pan/en_US/events/xdr-post-event.html
181+ Disallow: /content/pan/en_US/atlassian-domain-verification.html
182+ Disallow: /content/pan/en_US/documentation/
183+ Disallow: /documentation/
184+ Disallow: /company/in-the-news/2019/
185+ Disallow: /company/in-the-news/2018/
186+ Disallow: /company/in-the-news/2017/
187+ Disallow: /company/in-the-news/2016/
188+ Disallow: /company/in-the-news/2015/
189+ Disallow: /company/in-the-news/2014/
190+ Disallow: /company/in-the-news/2013/
191+ Disallow: /company/in-the-news/2012/
192+ Disallow: /company/in-the-news/2011/
193+ Disallow: /company/in-the-news/2010/
194+ Disallow: /company/in-the-news/2009/
195+ Disallow: /company/in-the-news/2008/
196+ Disallow: /company/in-the-news/2007/
197+ Disallow: /company/press/2019/
198+ Disallow: /company/press/2018/
199+ Disallow: /company/press/2017/
200+ Disallow: /company/press/2016/
201+ Disallow: /company/press/2015/
202+ Disallow: /company/press/2014/
203+ Disallow: /company/press/2013/
204+ Disallow: /company/press/2012/
205+ Disallow: /company/press/2011/
206+ Disallow: /company/press/2010/
207+ Disallow: /company/press/2009/
208+ Disallow: /company/press/2008/
209+ Disallow: /company/press/2007/
210+ Disallow: /securityevent/en_US_LV2.html
211+ Disallow: /content/pan/en_US/google992ef19868c7ad5e.html
212+ Disallow: /google992ef19868c7ad5e.html
213+ Disallow: /content/pan/en_US/.well_known/
214+ Disallow: /.well_known/
215+ Disallow: /events/future-of-security-revealed
216+ Disallow: /events/future-of-security-revealed-amer
217+ Disallow: /newbrand
218+ Disallow: /blog/wp-admin/
219+ Disallow: /devsectalks/wp-admin/
220+ Disallow: /perspectives/wp-admin/