User-agent: * Disallow: /api Disallow: /sys Disallow: /cache/ad Disallow: /.htaccess Disallow: /my Disallow: /go Disallow: /te