# All robots will spider the domain User-agent: * Disallow: # Disallow directory /images/ User-agent: * Disallow: /images/ # Disallow directory /download/ User-agent: * Disallow: /download/ # Disallow directory /flash/ User-agent: * Disallow: /flash/ # Disallow directory /header/ User-agent: * Disallow: /header/ # Disallow directory /stats/ User-agent: * Disallow: /stats/ # Disallow directory /product/ User-agent: * Disallow: /product/ # Disallow Files User-agent: * Disallow: /404error.htm Disallow: /GOOGLE7099f20c9771fed9.html Disallow: /fsgba0149e06da89c0982789121d96dce07.txt Disallow: /report.txt