NeuralCrawl

πŸ‡ΊπŸ‡Έ WordPress

wordpress.com · Top 1000 websites · rank #73 · Web · live robots.txt ↗

AI crawler access (latest snapshot, 5h ago)

blocked restricted allowed faded = inherited from the * wildcard group

GPTBot
ChatGPT-User
OAI-SearchBot
ClaudeBot
Claude-User
Claude-SearchBot
anthropic-ai
Claude-Web
CCBot
Google-Extended
Applebot-Extended
PerplexityBot
Perplexity-User
Bytespider
Amazonbot
FacebookBot
meta-externalagent
Meta-WebIndexer
meta-externalfetcher
cohere-ai
AI2Bot
Diffbot
omgili
YouBot
DuckAssistBot
MistralAI-User
PanguBot
Timpibot

Current robots.txt 1387 bytes · sha256 a3b39e93e899 · raw

# If you are regularly crawling WordPress.com sites, please use our firehose to receive real-time push updates instead.
# Please see https://developer.wordpress.com/docs/firehose/ for more details.

Sitemap: https://wordpress.com/sitemap.xml
Sitemap: https://wordpress.com/news-sitemap.xml

Sitemap: https://wordpress.com/go/sitemap.xml
Sitemap: https://wordpress.com/blog/sitemap.xml
Sitemap: https://wordpress.com/support/sitemap.xml

User-agent: *
Disallow: /wp-admin/
Allow: /wp-admin/admin-ajax.php
Disallow: /typo/?subdomain=
Disallow: /read/
Disallow: /*/read/
Allow: /forums/topic-tag/reader/
Allow: /*/forums/topic-tag/reader/
Allow: /support/reader/
Allow: /*/support/reader/
Disallow: /reader/
Disallow: /*/reader/
Allow: /log-in/
Allow: /log-in$
Disallow: /log-in?
Disallow: /abuse/?*
Disallow: /abuse?*
Disallow: /plugins/?s=
Disallow: /*/plugins/?s=
Disallow: /*?aff=
Disallow: /*&aff=
Disallow: /*?affiliate=
Disallow: /*&affiliate=
Disallow: /*?cid=
Disallow: /*&cid=
Disallow: /*?irclickid=
Disallow: /*&irclickid=
Disallow: /*/?like_comment=
Disallow: /*?retry=
Disallow: /*?sid=
Disallow: /*?action=
Disallow: /wp-login.php
Disallow: /wp-signup.php
Disallow: /press-this.php
Disallow: /remote-login.php
Disallow: /activate/
Disallow: /cgi-bin/
Disallow: /mshots/v1/
Disallow: /next/
Disallow: /public.api/

# This file was generated on Mon, 24 Aug 2026 06:52:06 +0000

Compare any two snapshots

Change history

  1. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  2. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  3. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  4. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  5. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  6. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  7. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  8. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  9. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  10. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  11. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  12. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  13. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  14. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  15. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  16. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  17. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  18. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  19. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  20. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  21. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  22. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  23. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  24. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  25. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  26. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  27. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  28. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  29. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  30. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  31. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  32. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  33. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  34. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  35. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  36. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  37. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  38. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  39. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  40. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  41. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  42. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  43. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  44. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  45. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  46. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  47. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  48. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  49. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  50. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  51. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  52. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  53. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  54. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  55. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  56. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  57. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  58. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  59. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  60. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  61. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  62. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  63. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  64. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  65. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  66. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  67. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  68. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  69. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  70. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  71. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  72. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  73. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  74. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  75. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  76. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  77. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  78. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  79. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  80. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  81. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  82. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  83. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  84. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  85. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  86. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  87. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  88. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  89. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  90. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  91. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  92. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  93. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  94. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  95. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  96. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  97. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  98. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  99. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  100. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  101. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  102. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  103. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  104. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  105. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  106. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  107. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  108. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  109. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  110. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  111. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  112. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  113. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  114. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  115. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  116. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  117. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  118. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  119. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  120. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  121. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  122. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  123. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  124. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  125. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  126. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  127. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  128. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  129. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  130. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  131. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  132. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  133. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  134. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  135. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  136. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  137. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  138. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  139. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  140. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  141. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  142. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  143. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  144. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  145. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  146. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  147. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  148. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  149. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  150. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  151. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  152. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  153. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  154. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  155. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  156. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  157. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  158. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  159. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  160. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  161. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  162. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  163. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  164. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  165. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  166. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  167. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  168. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  169. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  170. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  171. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  172. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  173. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  174. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  175. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  176. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  177. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  178. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  179. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  180. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  181. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  182. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  183. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  184. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  185. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  186. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  187. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  188. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  189. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  190. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  191. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  192. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  193. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  194. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  195. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  196. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  197. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  198. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  199. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  200. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  201. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  202. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  203. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  204. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  205. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  206. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  207. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  208. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  209. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  210. modified +1 −1 view diff → snapshot
    • Formatting/comment-only change (no semantic difference)
  211. initial snapshot
    • First snapshot of robots.txt archived