# # # NoQuarterArms.com Robots.txt File # # # # Changelog: # 07-06-07 set basic instructions to disallow indexing of construction directories # ## Rules for All Search Agents! ## User-agent: * # Block access to directories that don't output content Disallow: /css/ Disallow: /db/ Disallow: /lib/ # Block access to stats Disallow: /webalizer/ ## Individual Search Engines ## # Tell MSN and Slurp to index slowly User-Agent: msnbot Crawl-Delay: 30 User-Agent: Slurp Crawl-Delay: 30