# Robots.txt file # # All robots will spider the domain User-agent: * Disallow: /admin/ Disallow: /images/ Disallow: /bin/ Disallow: /old_site/ Disallow: /private/ Disallow: /aspnet_client/ Disallow: /*.axd