# robots.txt, nytimes.com 1/18/2001 # User-agent: * Disallow: /