Webz.io / robots.txt
← back to webz.io · change detected 2mo ago (2026-08-11T15:08:50Z) · modified +7 −0
What changed
- Crawl-delay for *: none → 10
- Added sitemap http://webz.io/sitemap_index.xml
Diff old (2026-07-23T14:11:06Z) → new (2026-08-11T15:08:50Z)
| @@ -1,2 +1,9 @@ | ||
| 1 | + Crawl-delay: 10 | |
| 2 | + # START YOAST BLOCK | |
| 3 | + # --------------------------- | |
| 1 | 4 | User-agent: * |
| 2 | 5 | Disallow: |
| 6 | + | |
| 7 | + Sitemap: http://webz.io/sitemap_index.xml | |
| 8 | + # --------------------------- | |
| 9 | + # END YOAST BLOCK |