The Baidu search engine has a voracious appetite for content and crawls one of my sites aggressively. It’s bad enough having to deal with load generated by bots from large technology companies with vast resources, but it’s another thing entirely when those bots crawl from dozens of IP addresses simultaneously and routinely browse thousands of […]