# Indicate which parts of the site should not be vistsed by a robot, # by providing a specially formatted file http://.../robots.txt User-agent: * Disallow: /cgi-bin/ Disallow: /tmp/ Disallow: /temp/ Disallow: /stat/