NeuralCrawl

Fortra / robots.txt snapshot

← back to fortra.com · fetched 2026-06-20T14:27:25Z (7h ago) · HTTP 200 · 2027 bytes · sha256 e64bb3bb92a7f533 · raw

final URL: https://www.fortra.com/robots.txt

1Sitemap: https://www.fortra.com/sitemap.xml
2
3User-agent: *
4# CSS, JS, Images
5Allow: /core/*.css$
6Allow: /core/*.css?
7Allow: /core/*.js$
8Allow: /core/*.js?
9Allow: /core/*.gif
10Allow: /core/*.jpg
11Allow: /core/*.jpeg
12Allow: /core/*.png
13Allow: /core/*.svg
14Allow: /profiles/*.css$
15Allow: /profiles/*.css?
16Allow: /profiles/*.js$
17Allow: /profiles/*.js?
18Allow: /profiles/*.gif
19Allow: /profiles/*.jpg
20Allow: /profiles/*.jpeg
21Allow: /profiles/*.png
22Allow: /profiles/*.svg
23# Directories
24Disallow: /core/
25Disallow: /profiles/
26# Files
27Disallow: /README.md
28Disallow: /composer/Metapackage/README.txt
29Disallow: /composer/Plugin/ProjectMessage/README.md
30Disallow: /composer/Plugin/Scaffold/README.md
31Disallow: /composer/Plugin/VendorHardening/README.txt
32Disallow: /composer/Template/README.txt
33Disallow: /modules/README.txt
34Disallow: /sites/README.txt
35Disallow: /themes/README.txt
36Disallow: /web.config
37# Paths (clean URLs)
38Disallow: /admin/
39Disallow: /comment/reply/
40Disallow: /filter/tips
41Disallow: /node/add/
42Disallow: /search/
43Disallow: /user/register
44Disallow: /user/password
45Disallow: /user/login
46Disallow: /user/logout
47Disallow: /media/oembed
48Disallow: /*/media/oembed
49# Paths (no clean URLs)
50Disallow: /index.php/admin/
51Disallow: /index.php/comment/reply/
52Disallow: /index.php/filter/tips
53Disallow: /index.php/node/add/
54Disallow: /index.php/search/
55Disallow: /index.php/user/password
56Disallow: /index.php/user/register
57Disallow: /index.php/user/login
58Disallow: /index.php/user/logout
59Disallow: /index.php/media/oembed
60Disallow: /index.php/*/media/oembed
61
62#
63# Upstream Robots Additions
64#
65# Block specific parameters
66Disallow: /*?__hsfp
67Disallow: /*?f=
68Disallow: /*?wtime
69
70# Allow specific parameters
71Allow: /*?page=
72
73# Resources URLs
74Disallow: /*/resources?
75Disallow: /*/resources$
76Disallow: /resources?
77Disallow: /resources$
78Allow: /resources/
79Allow: /*/resources/
80
81# Search URLs
82Disallow: /*/search
83Disallow: /search
84
85# Blog URLs
86Disallow: /*/blog?
87Disallow: /*/blog$
88Disallow: /blog?
89Disallow: /blog$
90Allow: /*/blog/
91Allow: /blog/