User-agent: YandexBot
User-agent: Ezooms
Disallow: /

User-agent: Blekkobot
Crawl-delay: 3

User-agent: msnbot
Crawl-delay: 3

User-agent: CCBot
Crawl-Delay: 3

User-agent: Baiduspider
Crawl-Delay: 3

User-agent: Slurp
Crawl-Delay: 3


Allow: /crd_images/

Sitemap: http://www.metmuseum.org/sitemaps/sitemap_index