GitHub / robots.txt
← back to github.com · change detected 18d ago (2026-10-20T08:19:06Z) · modified +2 −0
What changed
- Added sitemap https://github.com/sitemap_marketing.xml
Diff old (2026-10-17T01:09:51Z) → new (2026-10-20T08:19:06Z)
| @@ -1,5 +1,7 @@ | ||
| 1 | 1 | # If you would like to crawl GitHub contact us via https://support.github.com?tags=dotcom-robots |
| 2 | 2 | # We also provide an extensive API: https://docs.github.com |
| 3 | + Sitemap: https://github.com/sitemap_marketing.xml | |
| 4 | + | |
| 3 | 5 | User-agent: GPTBot |
| 4 | 6 | User-agent: OAI-SearchBot |
| 5 | 7 | User-agent: ClaudeBot |