NeuralCrawl

WordPress / robots.txt snapshot

← back to wordpress.com · fetched 2026-06-26T13:23:15Z (3h ago) · HTTP 200 · 1387 bytes · sha256 029e274d661ecd8a · raw

final URL: https://wordpress.com/robots.txt

1# If you are regularly crawling WordPress.com sites, please use our firehose to receive real-time push updates instead.
2# Please see https://developer.wordpress.com/docs/firehose/ for more details.
3
4Sitemap: https://wordpress.com/sitemap.xml
5Sitemap: https://wordpress.com/news-sitemap.xml
6
7Sitemap: https://wordpress.com/go/sitemap.xml
8Sitemap: https://wordpress.com/blog/sitemap.xml
9Sitemap: https://wordpress.com/support/sitemap.xml
10
11User-agent: *
12Disallow: /wp-admin/
13Allow: /wp-admin/admin-ajax.php
14Disallow: /typo/?subdomain=
15Disallow: /read/
16Disallow: /*/read/
17Allow: /forums/topic-tag/reader/
18Allow: /*/forums/topic-tag/reader/
19Allow: /support/reader/
20Allow: /*/support/reader/
21Disallow: /reader/
22Disallow: /*/reader/
23Allow: /log-in/
24Allow: /log-in$
25Disallow: /log-in?
26Disallow: /abuse/?*
27Disallow: /abuse?*
28Disallow: /plugins/?s=
29Disallow: /*/plugins/?s=
30Disallow: /*?aff=
31Disallow: /*&aff=
32Disallow: /*?affiliate=
33Disallow: /*&affiliate=
34Disallow: /*?cid=
35Disallow: /*&cid=
36Disallow: /*?irclickid=
37Disallow: /*&irclickid=
38Disallow: /*/?like_comment=
39Disallow: /*?retry=
40Disallow: /*?sid=
41Disallow: /*?action=
42Disallow: /wp-login.php
43Disallow: /wp-signup.php
44Disallow: /press-this.php
45Disallow: /remote-login.php
46Disallow: /activate/
47Disallow: /cgi-bin/
48Disallow: /mshots/v1/
49Disallow: /next/
50Disallow: /public.api/
51
52# This file was generated on Fri, 26 Jun 2026 09:53:59 +0000