Google ignores crawl-delay in robots.txt. Instead, they force you to register your site in webmaster tools, so you can set a custom crawl rate. This is then RESET to the default after 90 days, so at that time you have to login again and change it again! How fucking arrogant is that?
Other search engines respect crawl-delay, imagine if they all wanted us to create accounts and login every 90 days to stop their bots hammering our servers?
