NeuralCrawl

DXC Technology / robots.txt snapshot

← back to dxc.com · fetched 2026-08-17T17:05:52Z (2mo ago) · HTTP 200 · 2237 bytes · sha256 f659d1b284e040f9 · raw

final URL: https://dxc.com/robots.txt

1User-agent: *
2Allow: /
3Disallow: */search?search=
4Disallow: */search$
5
6#All the regional pages excluding homepages are disallowed for indexing
7Disallow: */language-masters/
8Disallow: */za/en/
9Disallow: */tw/zh/
10Disallow: */vn/en/
11Disallow: */tn/en/
12Disallow: */th/en/
13Disallow: */sk/sk/
14Disallow: */sg/en/
15Disallow: */se/en/
16Disallow: */pl/en/
17Disallow: */sa/en/
18Disallow: */qa/en/
19Disallow: */ph/en/
20Disallow: */no/en/
21Disallow: */nl/en/
22Disallow: */ng/en/
23Disallow: */my/en/
24Disallow: */ma/en/
25Disallow: */kr/en/
26Disallow: */lu/en/
27Disallow: */lt/en/
28Disallow: */ie/en/
29Disallow: */id/en/
30Disallow: */hr/en/
31Disallow: */hu/en/
32Disallow: */hk/en/
33Disallow: */fi/en/
34Disallow: */es/ca/
35Disallow: */eg/en/
36Disallow: */dk/da/
37Disallow: */dk/en/
38Disallow: */cz/en/
39Disallow: */cn/zh/
40Disallow: */ch/en/
41Disallow: */bg/en/
42Disallow: */be/nl/
43Disallow: */uk/en/
44Disallow: */pt/pt/
45Disallow: */pl/pl/
46Disallow: */pe/es/
47Disallow: */pa/es/
48Disallow: */nz/en/
49Disallow: */lu/fr/
50Disallow: */mx/es/
51Disallow: */jp/ja/
52Disallow: */it/it/
53Disallow: */in/en/
54Disallow: */fr/fr/
55Disallow: */es/es/
56Disallow: */ec/es/
57Disallow: */de/de/
58Disallow: */cr/es/
59Disallow: */co/es/
60Disallow: */ca/fr/
61Disallow: */cl/es/
62Disallow: */ca/en/
63Disallow: */br/pt/
64Disallow: */be/fr/
65Disallow: */be/en/
66Disallow: */au/en/
67Disallow: */ae/en/
68Disallow: */at/de/
69Disallow: */ar/es/
70Disallow: */ar/es/
71Disallow: */sk/en/
72
73Allow: */au/en/careers/graduate-program$
74Allow: */au/en/cp/claims-management-services$
75Allow: */au/en/cp/claims-management-services/new-south-wales$
76Allow: */au/en/cp/claims-management-services/new-south-wales/worker$
77Allow: */au/en/cp/claims-management-services/victoria$
78Allow: */au/en/cp/claims-management-services/victoria/employer$
79Allow: */nz/en/careers/graduate-program$
80Allow: */uk/en/careers/early-careers$
81
82
83#OpenAI bot for indexing web content for ChatGPT search
84User-agent: OAI-SearchBot
85Allow: /
86
87#Identifies user requests from ChatGPT
88User-agent: ChatGPT-User
89Allow: /
90
91#Bot from Perplexity.ai to fetch real-time info with citations
92User-agent: PerplexityBot
93Allow: /
94
95#Exa.ai crawler for dev-focused AI search
96User-agent: ExaBot
97Allow: /
98
99
100
101Sitemap: https://dxc.com/sitemap.xml
102Sitemap: https://dxc.com/newsroom-sitemap.xml
103
104
105