NeuralCrawl

Glencore / robots.txt

← back to glencore.com · change detected 1mo ago (2026-09-14T14:12:31Z) · modified +3 −3

What changed

Diff old (2026-09-06T05:23:16Z) → new (2026-09-14T14:12:31Z)

@@ -3,9 +3,6 @@
33 
44  # Excluded documents
55  User-agent: *
6- Disallow: /.rest/api/v1/documents/static/610492b3-3d6e-4622-aff1-4fc0f747b271/Business+confirmation+general+terms-purchase.pdf
7- Disallow: /.rest/api/v1/documents/static/72f40bbd-1240-4e41-8177-5fba47c6323f/Business+confirmation+general+terms-purchase-ext.pdf
8- Disallow: /.rest/api/v1/documents/static/975c896c-6824-401f-9def-50bc0d31d097/Business+confirmation+general+terms-sale.pdf
96  Disallow: /.rest/api/v1/documents/599c9228f4ec49b3c54710ae10fe0df5/Couverture+des+cales+de+navire+lors+du+d%C3%A9chargement+du+concentr%C3%A9+de+nickel+%E2%80%93+%C3%89tude+de+pr%C3%A9faisabilit%C3%A9.pdf
107  Disallow: /.rest/api/v1/documents/dc41ac990a2bc1386966ec4a87ce9e0c/Coupa+Supplier+Portal+Onboarding+Guide.pdf
118  Disallow: /.rest/api/v1/documents/e11fccb4ea7bf956a9a8b067949b5d42/Guide+d%27int%C3%A9gration+-+Portail+Fournisseur+Coupa.pdf
@@ -106,3 +103,6 @@
106103  Disallow: /.rest/api/v1/documents/static/551d1418-9701-46e0-a4c4-f335433a4716/Supplier+Code+of+Conduct+-+Mandarin.pdf
107104  Disallow: /.rest/api/v1/documents/f783ed07f6529634aae7d2801136327d/GLT+3286ZI+IsaMill+Broch+8pp+A4+2026-03+%28ESP%29+01-SR.pdf
108105  Disallow: /.rest/api/v1/documents/7a5d3eb7a056308363da09ab224f03a9/GLT+3286A+IsaMill+Broch+8pp+A4+2026-03+%28ENG%29+02-SR.pdf
106+ Disallow: /.rest/api/v1/documents/static/610492b3-3d6e-4622-aff1-4fc0f747b271/Business%2Bconfirmation%2Bgeneral%2Bterms-purchase.pdf
107+ Disallow: /.rest/api/v1/documents/static/72f40bbd-1240-4e41-8177-5fba47c6323f/Business%2Bconfirmation%2Bgeneral%2Bterms-purchase-ext.pdf
108+ Disallow: /.rest/api/v1/documents/static/975c896c-6824-401f-9def-50bc0d31d097/Business%2Bconfirmation%2Bgeneral%2Bterms-sale.pdf