# www.robotstxt.org/ User-agent: * Disallow: /api/* Disallow: *?view_only= crawl-delay: 10 # Robots that have misbehaved User-agent: PingBot User-agent: PerplexityBot User-agent: GPTBot User-agent: BaiduSpider User-agent: Meta-ExternalAgent Disallow: *