# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-Agent: * # Disallow: / # Google, Yahoo, MSN et al all seem to be trying to index this (non-existant) url. God knows why # Tell them to sod off User-Agent: * Disallow: /~magnolia/