Back in 2016 Brad compared Varnish and Nginx FastCGI Caching to see which would come out on top. Back then Nginx FastCGI Caching was superior in performance, but has anything changed in the last two years? That’s what we’re here to find out!
This time round I’m also going to benchmark WordPress without any cache and throw a WordPress caching plugin into the mix. I’ve opted for Simple Cache, because as the name suggests it’s simple to set up. Let’s get to it!
Quite a lot actually. First, there’s been a number of software releases over the last 2 years – the majority of which all promise performance improvements. But do those performance improvements equate to more requests per second and knock FastCGI caching off the top spot? We’ll find out later!
Second, HTTPS is everywhere thanks mostly to Let’s Encrypt, and that’s a good thing! A secure web is a better web, but what about HTTPS performance? HTTPS has always had a bad rep when it comes to performance, because the TLS handshake is more CPU intensive. In this article I’m only going to benchmark HTTPS, because it’s 2018 and every site should be using it!
In the past when performing benchmarks on the Delicious Brains blog, our go-to tool has always been Blitz.io. Unfortunately the service is no longer accepting new subscriptions, so we needed to find an alternative tool. Enter ApacheBench.
ApacheBench is a CLI tool designed by the Apache Software Foundation and was originally created for testing how well Apache performs.
All of the tests in this article will be performed using the same options, except for testing WordPress with no cache configured. The reason for this is because sending that many concurrent requests directly to PHP and MySQL will do nothing but increase the average response time, which isn’t a fair representation of PHP 7.1s performance.
ab -n 10000 -c 100 https://siteunderload.com/
This simulates 10,000 requests, with a concurrency of 100 requests. Meaning, ApacheBench will send a total of 10,000 requests in batches of 100 at a time. For the non-cached version I will use a concurrency of 20 requests.
Running ApacheBench will output results similar to below:
In this article we’re mostly concerned with Requests per second and Connection Times. Each test will be performed a total of 10 times and the average used for comparison.
The Server Stack
All benchmarks will utilize the same server stack, which consists of the following software:
- Digital Ocean 2GB ($10/mo)
- Ubuntu 16.04.4
- PHP 7.1.15
- Nginx 1.13.6
- MariaDB 10.2.13
- Varnish 6.0
- WordPress 4.9.4, Twenty Seventeen
Varnish will be completely disabled when not needed for the current set of benchmarks. Nginx will be used to terminate HTTPS requests, because Varnish is unable to do so. This will result in the following setup:
Nginx:443 > Varnish:80 > Nginx:8080
There are other proxies available to perform the HTTPS termination, but I’ve opted for Nginx to keep the number of moving parts to a minimum.
Requests Per Second
As expected, WordPress doesn’t perform well, which is due to the bottleneck created by the database server. Simply taking the database server out of the equation by enabling a page cache gives a 10x increase in requests.
Average Response Time (ms)
A high requests per second doesn’t mean much if those requests are slow to complete, that’s why it’s important to also measure response time. The average response time is the total time it takes for a request to complete.
When a server is under heavy load the average response time will usually increase. This happens because the server is only able to handle a certain number of concurrent connections. When this number is exceeded, all additional connections will be queued and processed as resources become available.
Generally, the fewer moving parts you have in a request lifecycle, the lower the average response time will be. That’s why Nginx FastCGI caching performs so well, because it only has to serve a static file from disk (which will likely be cached in memory due to the Linux Page Cache). WordPress hits Nginx, PHP and the database server, whereas caching plugins only hit Nginx and PHP.
Nginx FastCGI Cache is the clear winner when it comes to outright performance. It’s not only able to handle more requests per second, but also serve each request 55ms quicker on average. If you’re interested in seeing the full results you can view them here.
Performance isn’t the only consideration though, what about ease of configuration? WordPress plugins generally come out on top, excluding W3 Total Cache (honestly, there’s more to life than sifting through that control panel). Although Nginx FastCGI caching is relatively easy to configure, it requires sudo privileges on the server. Varnish on the other hand is far more complex to set up due to the requirement for HTTPS termination.
If Varnish isn’t the quickest solution and the most difficult to setup, why on earth would you opt for it? Quite simply, Varnish is still the best at handling more complex cache invalidation rules. But for most sites this isn’t a requirement. Here at Delicious Brains we opt for a longer cache duration (7 days) and purge the entire cache on content update. We’ve found this solution way more reliable, versus trying to determine exactly which pages should be purged from the cache, which can get complicated pretty quickly due to post archives and pagination.
With all that taken into consideration, Nginx FastCGI caching is still our preferred caching solution, which is why we’ve removed Varnish from our server and rely purely on Nginx.
What do you use for caching your WordPress sites? Let us know in the comments below.