Webz.io / robots.txt
← back to webz.io · change detected 2mo ago (2026-08-17T11:01:05Z) · modified +0 −1
What changed
- Crawl-delay for *: 10 → none
Diff old (2026-08-11T15:08:50Z) → new (2026-08-17T11:01:05Z)
| @@ -1,4 +1,3 @@ | ||
| 1 | - Crawl-delay: 10 | |
| 2 | 1 | # START YOAST BLOCK |
| 3 | 2 | # --------------------------- |
| 4 | 3 | User-agent: * |