NeuralCrawl

Conagra Brands / robots.txt snapshot

← back to conagrabrands.com · fetched 2026-08-27T14:17:26Z (2mo ago) · HTTP 200 · 1359 bytes · sha256 867371713a34600a · raw

final URL: https://www.conagrabrands.com/robots.txt

1User-agent: *
2# CSS, JS, Images
3Allow: /core/*.css$
4Allow: /core/*.css?
5Allow: /core/*.js$
6Allow: /core/*.js?
7Allow: /core/*.gif
8Allow: /core/*.jpg
9Allow: /core/*.jpeg
10Allow: /core/*.png
11Allow: /core/*.svg
12Allow: /profiles/*.css$
13Allow: /profiles/*.css?
14Allow: /profiles/*.js$
15Allow: /profiles/*.js?
16Allow: /profiles/*.gif
17Allow: /profiles/*.jpg
18Allow: /profiles/*.jpeg
19Allow: /profiles/*.png
20Allow: /profiles/*.svg
21# Directories
22Disallow: /core/
23Disallow: /profiles/
24# Files
25Disallow: /README.txt
26Disallow: /web.config
27# Paths (clean URLs)
28Disallow: /admin/
29Disallow: /comment/reply/
30Disallow: /filter/tips/
31Disallow: /node/add/
32Disallow: /search/
33Disallow: /user/register/
34Disallow: /user/password/
35Disallow: /user/login/
36Disallow: /user/logout/
37Disallow: /my/
38# Paths (no clean URLs)
39Disallow: /index.php/admin/
40Disallow: /index.php/comment/reply/
41Disallow: /index.php/filter/tips/
42Disallow: /index.php/node/add/
43Disallow: /index.php/search/
44Disallow: /index.php/user/password/
45Disallow: /index.php/user/register/
46Disallow: /index.php/user/login/
47Disallow: /index.php/user/logout/
48Disallow: /index.php/my/
49# Custom
50Disallow: /node/
51Disallow: /taxonomy/
52Disallow: /advanced-search
53Disallow: /es/node/
54Disallow: /es/taxonomy/
55Disallow: /es/busqueda-avanzada
56Disallow: /tags/*
57Disallow: /es/advanced-search
58
59Sitemap: https://www.conagrabrands.com/sitemap.xml