Threads / robots.txt
← back to threads.net · change detected 8d ago (2026-10-30T05:39:55Z) · modified +14 −0
What changed
- Added rules for user-agent ‘redditbot’
Diff old (2026-10-02T01:50:39Z) → new (2026-10-30T05:39:55Z)
| @@ -349,6 +349,20 @@ | ||
| 349 | 349 | Disallow: /query/ |
| 350 | 350 | Disallow: /settings/ |
| 351 | 351 | |
| 352 | + User-agent: redditbot | |
| 353 | + Disallow: /accounts/ | |
| 354 | + Disallow: /activity/ | |
| 355 | + Disallow: /ajax/ | |
| 356 | + Disallow: /client_error/ | |
| 357 | + Disallow: /deck/ | |
| 358 | + Disallow: /intent/ | |
| 359 | + Disallow: /logging/ | |
| 360 | + Disallow: /onboarding/ | |
| 361 | + Disallow: /publicapi/ | |
| 362 | + Disallow: /qp/batch_fetch_web/ | |
| 363 | + Disallow: /query/ | |
| 364 | + Disallow: /settings/ | |
| 365 | + | |
| 352 | 366 | User-agent: * |
| 353 | 367 | Disallow: / |
| 354 | 368 |