NeuralCrawl

Fonterra / robots.txt snapshot

← back to fonterra.com · fetched 2026-06-20T01:10:30Z (15h ago) · HTTP 200 · 591 bytes · sha256 74ca153446215ba0 · raw

final URL: https://www.fonterra.com/robots.txt

1User-Agent: *
2Disallow: /nz/en/redirection.html
3Disallow: /nz/en/redirection/careers-myfonterra.html
4Disallow: /nz/en/redirection/farmsourcecareers.html
5Disallow: /nz/en/redirection/spreadtheword.html
6Disallow: /nz/en/latest-news.html
7Disallow: /nz/en/latest-news/news-article-1.html
8Disallow: /nz/en/latest-news/news-article-2.html
9Disallow: /nz/en/latest-news/news-article-3.html
10Disallow: /nz/en/latest-news/news-article-4.html
11Disallow: /search/_jcr_content/responsivegrid/
12Disallow: /*article-template-new*
13Disallow: /*media-template-new*
14
15Sitemap: https://www.fonterra.com/sitemap.xml