# All robots will spider the domain User-agent: * Disallow: /cgi-bin/ Disallow: /images/ Disallow: /slog/ Disallow: /search/ Disallow: /sold/ Disallow: /aspnet_client/ Disallow: /cp/ Disallow: /webalizer/ Disallow: /_private/ Disallow: /_vti_cnf/ Disallow: /_vti_log/ Disallow: /_vti_pvt/ Disallow: /_vti_script/ Disallow: /_vti_txt/