Rate Limiting Baiduspider Using nginx

The Baidu search engine has a voracious appetite for content and crawls one of my sites aggressively. It’s bad enough having to deal with load generated by bots from large technology companies with vast resources, but it’s another thing entirely when those bots crawl from dozens of IP addresses simultaneously and routinely browse thousands of […]

Using systemd Timers to Renew Let’s Encrypt Certificates

This is a quick blog post to share the systemd timers that I use to automate the renewal of my Let’s Encrypt certificates. I prefer systemd timers to cron jobs for task scheduling because they are more flexible and easier to debug. I assume that you know what Let’s Encrypt is and that you already […]

Finding the Nginx gzip_comp_level Sweet Spot

When a web server creates a web page it generally pulls loads of crap from the database/disk and stitches together some ginormous hodgepodge of raw markup hundreds or thousands of kilobytes in size. As HTML is literally just a heap of embarrassingly compressible plaintext, turning on nginx’s gzip compression filter is a no-brainer. gzip on; […]

Maps and Custom Error Pages in Nginx

During a recent web application upgrade I had to limit access to the the web servers; I wanted the administrators and myself to be able to access the site, but for everyone else to see an “Under Construction” page. My initial plan was to test if the $remote_addr was one of the allowed IPs, and […]