User-agent: * Disallow: /search Disallow: /api Disallow: /admin/ Crawl-delay: 10