NeuralCrawl

Exa / robots.txt snapshot

← back to exa.ai · fetched 2026-10-21T02:33:48Z (17d ago) · HTTP 200 · 1816 bytes · sha256 fd449680ad8a5125 · raw

final URL: https://exa.ai/robots.txt

1User-Agent: *
2Allow: /library$
3Allow: /library/$
4Allow: /library?
5Allow: /library/person$
6Allow: /library/person/$
7Allow: /library/person?
8Allow: /library/organization$
9Allow: /library/organization/$
10Allow: /library/organization?
11Allow: /library/place$
12Allow: /library/place/$
13Allow: /library/place?
14Allow: /library/publication$
15Allow: /library/publication/$
16Allow: /library/publication?
17Allow: /library/legal/opinion$
18Allow: /library/legal/opinion/$
19Allow: /library/legal/opinion?
20Allow: /library/legal/patent$
21Allow: /library/legal/patent/$
22Allow: /library/legal/patent?
23Allow: /library/podcast$
24Allow: /library/podcast/$
25Allow: /library/podcast?
26Allow: /library/markets/stocks$
27Allow: /library/markets/stocks/$
28Allow: /library/markets/stocks?
29Allow: /library/markets$
30Allow: /library/markets/$
31Allow: /library/markets?
32Allow: /library/sports$
33Allow: /library/sports/$
34Allow: /library/sports?
35Allow: /library/weather$
36Allow: /library/weather/$
37Allow: /library/weather?
38Allow: /library/indicator$
39Allow: /library/indicator/$
40Allow: /library/indicator?
41Allow: /library/geoarea$
42Allow: /library/geoarea/$
43Allow: /library/geoarea?
44Allow: /library/earnings-call$
45Allow: /library/earnings-call/$
46Allow: /library/earnings-call?
47Allow: /library/job-posting$
48Allow: /library/job-posting/$
49Allow: /library/job-posting?
50Allow: /library/product$
51Allow: /library/product/$
52Allow: /library/product?
53Allow: /library/event$
54Allow: /library/event/$
55Allow: /library/event?
56Allow: /library/wikipedia-article$
57Allow: /library/wikipedia-article/$
58Allow: /library/wikipedia-article?
59Allow: /library/sec-filing$
60Allow: /library/sec-filing/$
61Allow: /library/sec-filing?
62Disallow: /search?*
63Disallow: /hackathon/
64Disallow: /library/*/
65Disallow: /library/places
66
67Sitemap: https://exa.ai/sitemap.xml
68Sitemap: https://exa.ai/docs/sitemap.xml