# See http://www.robotstxt.org/wc/norobots.html for documentation on how to use the robots.txt file # # To ban all spiders from the entire site uncomment the next two lines: # User-Agent: * # Disallow: / Disallow: /listing* Disallow: /listing_raw/* Disallow: /listing_images/* Disallow: /listing_thumbs/* Disallow: /listing_small_thumbs/* Disallow: /images/*