NeuralCrawl

Europe PMC / robots.txt snapshot

← back to europepmc.org · fetched 2026-07-16T17:46:03Z (20d ago) · HTTP 200 · 1137 bytes · sha256 6a9be575453ce6d9 · raw

final URL: https://europepmc.org/robots.txt

1# UKPMC robots.txt
2
3# Google
4User-agent: Googlebot
5Disallow: /classic
6#Disallow: /*/articles/*/pdf/
7#Disallow: /articles/*/pdf/
8#Disallow: /articles/*?pdf=render
9#Disallow: /articles/*?*blobtype=pdf*
10Disallow: /advancedsearch
11Disallow: /feedback/*
12Disallow: /*?wicket:interface*
13#Disallow: /search/*
14Disallow: /*?europe_pmc_extredirect=*
15
16# Yahoo
17User-agent: Slurp
18Disallow: /classic
19Disallow: /*/articles/*/pdf/
20Disallow: /articles/*/pdf/
21Disallow: /articles/*?pdf=render
22Disallow: /articles/*?*blobtype=pdf*
23Disallow: /advancedsearch
24Disallow: /feedback/*
25Disallow: /*?wicket:interface*
26Disallow: /search/*
27Disallow: /*?europe_pmc_extredirect=*
28
29# Bing
30User-agent: MSNbot
31Disallow: /classic
32Disallow: /*/articles/*/pdf/
33Disallow: /articles/*/pdf/
34Disallow: /articles/*?pdf=render
35Disallow: /articles/*?*blobtype=pdf*
36Disallow: /advancedsearch
37Disallow: /feedback/*
38Disallow: /*?wicket:interface*
39Disallow: /search/*
40Disallow: /*?europe_pmc_extredirect=*
41
42# For all other bots
43User-agent: *
44Disallow: /
45
46Sitemap: http://europepmc.org/sitemap_abstract_index.xml
47Sitemap: http://europepmc.org/sitemap_fulltext_index.xml
48Crawl-delay: 10
49
50