User-agent: * Disallow: /cgi-bin Disallow: /scgi-bin Disallow: /ct Disallow: /ct_dir Disallow: /c_us.html