DEV Community / robots.txt snapshot

← back to dev.to · fetched 2026-06-20T01:10:31Z (18h ago) · HTTP 200 · 450 bytes · sha256 983a578c850154f6 · raw

final URL: https://dev.to/robots.txt

1	User-agent: *
2	Disallow: //actions_panel
3	Disallow: /users/auth/twitter*
4	Disallow: /users/auth/github*
5	Disallow: /report-abuse?url=*
6	Disallow: /connect/@*
7	Disallow: /search?q=*
8	Disallow: /search/?q=*
9	Disallow: /search/feed_content?*
10	Disallow: /listings?q=
11	Disallow: /mod/*
12	Disallow: /mod?*
13	Disallow: /admin/*
14	Disallow: /reactions?*
15	Disallow: /async_info/base_data
16	Disallow: /ahoy/*
17	Disallow: /auth_pass/*
18
19	Sitemap: https://dev.to/sitemap-index.xml