NeuralCrawl

Glencore / robots.txt

← back to glencore.com · change detected 1mo ago (2026-09-24T10:55:22Z) · modified +22 −22

What changed

Diff old (2026-09-20T03:47:10Z) → new (2026-09-24T10:55:22Z)

@@ -3,28 +3,6 @@
33 
44  # Excluded documents
55  User-agent: *
6- Disallow: /.rest/api/v1/documents/599c9228f4ec49b3c54710ae10fe0df5/Couverture+des+cales+de+navire+lors+du+d%C3%A9chargement+du+concentr%C3%A9+de+nickel+%E2%80%93+%C3%89tude+de+pr%C3%A9faisabilit%C3%A9.pdf
7- Disallow: /.rest/api/v1/documents/dc41ac990a2bc1386966ec4a87ce9e0c/Coupa+Supplier+Portal+Onboarding+Guide.pdf
8- Disallow: /.rest/api/v1/documents/e11fccb4ea7bf956a9a8b067949b5d42/Guide+d%27int%C3%A9gration+-+Portail+Fournisseur+Coupa.pdf
9- Disallow: /.rest/api/v1/documents/535eca4046bad6d3dec0a5c608299fd0/Inscription+et+Formation+Portail+Fournisseur+Coupa-+8+septembre+2025.pdf
10- Disallow: /.rest/api/v1/documents/e33dab157f5c29da07a1873996829885/Informations+Importantes+-+Prochain+Lancement+de+Coupa+-+2+Juillet+2025+%281%29.pdf
11- Disallow: /.rest/api/v1/documents/2c4c65ce2237cc9d2efdd214ec1b85df/We+are+Live%21+_+November+3%2C+2025.pdf
12- Disallow: /.rest/api/v1/documents/1ab16a3be154f86e1478a8f0e6277f65/D%C3%A9ploiement+de+Coupa%21+_+3+novembre+2025.pdf
13- Disallow: /.rest/api/v1/documents/6f67350fd644d8a8e80cb968cb4e05cc/CSP+Training-+Sourcing%2C+Contracting+%26+Purchasing.pdf
14- Disallow: /.rest/api/v1/documents/f19a9c2d4af60ca783b00d48982f04ee/CSP+Training-+Invoicing.pdf
15- Disallow: /.rest/api/v1/documents/8ce8e52a3a36e7f46c627716d4be7a2f/CSP+Training-+CSP+Setup%2C+Navigation+and+Use.pdf
16- Disallow: /.rest/api/v1/documents/29270520d8811484274cdcc5b6a55d6d/Formation+CSP+-+Facturation.pdf
17- Disallow: /.rest/api/v1/documents/44ff436fe002f3f95a6748364b850ab2/Formation+CSP+-+Configuration%2C+Navigation+et+Utilisation.pdf
18- Disallow: /.rest/api/v1/documents/ec11f20e649d3868966aa0f735ad9a22/Formation+CSP+-+Approvisionnement%2C+Contrats+et+Bons+de+Commande+%281%29.pdf
19- Disallow: /.rest/api/v1/documents/9881dbb346fd16ad888307a9906ff1aa/Ligne+de+tir+-+protocole.pdf
20- Disallow: /.rest/api/v1/documents/90879a10ff0a59a8ba76162239eaeb85/Travaux+en+hauteur.pdf
21- Disallow: /.rest/api/v1/documents/e7bfa3067fa24b21269479b31991848d/Gestion+des+entrepreneurs.pdf
22- Disallow: /.rest/api/v1/documents/2c4be5bcf9bf5776ab8973ee6c1bd657/Travail+en+hauteur+%2802%29_outil.pdf
23- Disallow: /.rest/api/v1/documents/a6ea2e09ae9c60292364dcacb7c1e2e6/QUIZ+EXAMEN_GestionEntrepreneurs_H1913_R03.pdf
24- Disallow: /.rest/api/v1/documents/ceab977abd9d9b76e03daf10f2a9d8b9/H1913_Presentation.pdf
25- Disallow: /.rest/api/v1/documents/fead04d171ba6a1639a7027cf89bd06e/IsaMill+Design+Improvements+and+Operational+Performance+at+Anglo+Platinum+-+Rule%5B1%5D.pdf
26- Disallow: /.rest/api/v1/documents/459a0eac148b9736d631757eb1bfa594/Joe_Pease.pdf
27- Disallow: /.rest/api/v1/documents/23b7375a3c7cc913e5682ade249db895/CIM+2024_Significant+Potential+of+an+IsaMill%E2%84%A2+in+Secondary+Grinding+Duty_.pdf
286  Disallow: /.rest/api/v1/documents/static/644477c5-ca8a-4785-826b-1d86aa69f48b/%E4%BA%BA%E6%9D%83+-+Supplier+Code+of+Conduct+Training+%28CHI%29.pdf
297  Disallow: /.rest/api/v1/documents/static/635f225d-72e0-400e-a793-c936790160c5/%E5%81%A5%E5%BA%B7%E4%B8%8E%E5%AE%89%E5%85%A8+-+Supplier+Code+of+Conduct+Training+%28CHI%29.pdf
308  Disallow: /.rest/api/v1/documents/static/f6588712-bf12-4c8c-862c-fb550b789941/%E5%85%AC%E5%B9%B3%E4%B8%8E%E5%B0%8A%E9%87%8D+-+Supplier+Code+of+Conduct+Training+%28CHI%29.pdf
@@ -103,6 +81,28 @@
10381  Disallow: /.rest/api/v1/documents/static/551d1418-9701-46e0-a4c4-f335433a4716/Supplier+Code+of+Conduct+-+Mandarin.pdf
10482  Disallow: /.rest/api/v1/documents/f783ed07f6529634aae7d2801136327d/GLT+3286ZI+IsaMill+Broch+8pp+A4+2026-03+%28ESP%29+01-SR.pdf
10583  Disallow: /.rest/api/v1/documents/7a5d3eb7a056308363da09ab224f03a9/GLT+3286A+IsaMill+Broch+8pp+A4+2026-03+%28ENG%29+02-SR.pdf
84+ Disallow: /.rest/api/v1/documents/599c9228f4ec49b3c54710ae10fe0df5/Couverture+des+cales+de+navire+lors+du+d%C3%A9chargement+du+concentr%C3%A9+de+nickel+%E2%80%93+%C3%89tude+de+pr%C3%A9faisabilit%C3%A9.pdf
85+ Disallow: /.rest/api/v1/documents/dc41ac990a2bc1386966ec4a87ce9e0c/Coupa+Supplier+Portal+Onboarding+Guide.pdf
86+ Disallow: /.rest/api/v1/documents/e11fccb4ea7bf956a9a8b067949b5d42/Guide+d%27int%C3%A9gration+-+Portail+Fournisseur+Coupa.pdf
87+ Disallow: /.rest/api/v1/documents/535eca4046bad6d3dec0a5c608299fd0/Inscription+et+Formation+Portail+Fournisseur+Coupa-+8+septembre+2025.pdf
88+ Disallow: /.rest/api/v1/documents/e33dab157f5c29da07a1873996829885/Informations+Importantes+-+Prochain+Lancement+de+Coupa+-+2+Juillet+2025+%281%29.pdf
89+ Disallow: /.rest/api/v1/documents/2c4c65ce2237cc9d2efdd214ec1b85df/We+are+Live%21+_+November+3%2C+2025.pdf
90+ Disallow: /.rest/api/v1/documents/1ab16a3be154f86e1478a8f0e6277f65/D%C3%A9ploiement+de+Coupa%21+_+3+novembre+2025.pdf
91+ Disallow: /.rest/api/v1/documents/6f67350fd644d8a8e80cb968cb4e05cc/CSP+Training-+Sourcing%2C+Contracting+%26+Purchasing.pdf
92+ Disallow: /.rest/api/v1/documents/f19a9c2d4af60ca783b00d48982f04ee/CSP+Training-+Invoicing.pdf
93+ Disallow: /.rest/api/v1/documents/8ce8e52a3a36e7f46c627716d4be7a2f/CSP+Training-+CSP+Setup%2C+Navigation+and+Use.pdf
94+ Disallow: /.rest/api/v1/documents/29270520d8811484274cdcc5b6a55d6d/Formation+CSP+-+Facturation.pdf
95+ Disallow: /.rest/api/v1/documents/44ff436fe002f3f95a6748364b850ab2/Formation+CSP+-+Configuration%2C+Navigation+et+Utilisation.pdf
96+ Disallow: /.rest/api/v1/documents/ec11f20e649d3868966aa0f735ad9a22/Formation+CSP+-+Approvisionnement%2C+Contrats+et+Bons+de+Commande+%281%29.pdf
97+ Disallow: /.rest/api/v1/documents/9881dbb346fd16ad888307a9906ff1aa/Ligne+de+tir+-+protocole.pdf
98+ Disallow: /.rest/api/v1/documents/90879a10ff0a59a8ba76162239eaeb85/Travaux+en+hauteur.pdf
99+ Disallow: /.rest/api/v1/documents/e7bfa3067fa24b21269479b31991848d/Gestion+des+entrepreneurs.pdf
100+ Disallow: /.rest/api/v1/documents/2c4be5bcf9bf5776ab8973ee6c1bd657/Travail+en+hauteur+%2802%29_outil.pdf
101+ Disallow: /.rest/api/v1/documents/a6ea2e09ae9c60292364dcacb7c1e2e6/QUIZ+EXAMEN_GestionEntrepreneurs_H1913_R03.pdf
102+ Disallow: /.rest/api/v1/documents/ceab977abd9d9b76e03daf10f2a9d8b9/H1913_Presentation.pdf
103+ Disallow: /.rest/api/v1/documents/fead04d171ba6a1639a7027cf89bd06e/IsaMill+Design+Improvements+and+Operational+Performance+at+Anglo+Platinum+-+Rule%5B1%5D.pdf
104+ Disallow: /.rest/api/v1/documents/459a0eac148b9736d631757eb1bfa594/Joe_Pease.pdf
105+ Disallow: /.rest/api/v1/documents/23b7375a3c7cc913e5682ade249db895/CIM+2024_Significant+Potential+of+an+IsaMill%E2%84%A2+in+Secondary+Grinding+Duty_.pdf
106106  Disallow: /.rest/api/v1/documents/static/610492b3-3d6e-4622-aff1-4fc0f747b271/Business%2Bconfirmation%2Bgeneral%2Bterms-purchase.pdf
107107  Disallow: /.rest/api/v1/documents/static/72f40bbd-1240-4e41-8177-5fba47c6323f/Business%2Bconfirmation%2Bgeneral%2Bterms-purchase-ext.pdf
108108  Disallow: /.rest/api/v1/documents/static/975c896c-6824-401f-9def-50bc0d31d097/Business%2Bconfirmation%2Bgeneral%2Bterms-sale.pdf