NeuralCrawl

CORE / robots.txt

← back to core.ac.uk · change detected 2mo ago (2026-08-14T16:15:55Z) · modified +29 −0

What changed

Diff old (2026-08-13T21:57:57Z) → new (2026-08-14T16:15:55Z)

@@ -1,1 +1,30 @@
1+ # As a condition of accessing this website, you agree to abide by the following
2+ # content signals:
3+
4+ # (a) If a content-signal = yes, you may collect content for the corresponding
5+ # use.
6+ # (b) If a content-signal = no, you may not collect content for the
7+ # corresponding use.
8+ # (c) If the website operator does not include a content signal for a
9+ # corresponding use, the website operator neither grants nor restricts
10+ # permission via content signal with respect to the corresponding use.
11+
12+ # The content signals and their meanings are:
13+
14+ # search: building a search index and providing search results (e.g., returning
15+ # hyperlinks and short excerpts from your website's contents). Search does not
16+ # include providing AI-generated search summaries.
17+ # ai-input: inputting content into one or more AI models (e.g., retrieval
18+ # augmented generation, grounding, or other real-time taking of content for
19+ # generative AI search answers).
20+ # ai-train: training or fine-tuning AI models.
21+
22+ # ANY RESTRICTIONS EXPRESSED VIA CONTENT SIGNALS ARE EXPRESS RESERVATIONS OF
23+ # RIGHTS UNDER ARTICLE 4 OF THE EUROPEAN UNION DIRECTIVE 2019/790 ON COPYRIGHT
24+ # AND RELATED RIGHTS IN THE DIGITAL SINGLE MARKET.
25+
26+
27+ User-agent: *
28+ Disallow: /search
29+
130  Sitemap: https://core.ac.uk/sitemap.xml