# robot eXclusion principle User-agent: * Disallow: /cgi-bin/ Disallow: /derby/ Disallow: /~peterg Disallow: /stats/ # 6/6/04 - Disable access to "msnbot" User-agent: msnbot Disallow: /