Hosting WordPress Yourself Part 4 – Server Monitoring and Caching

#

In the previous installment of this series, I walked you through the process of how to quickly setup sites using WP-CLI. However, no considerations were made on how well the server would handle traffic under heavy load. As a system administrator, it is also imperative that you can quickly determine the current utilisation of system resources. This will better help you to gauge how many sites your server can potentially handle, and when you may need to plan the scaling of hardware or cloud resources. In this post I will guide you through the process of tackling all of those issues, as well as setting up server monitoring and alerts.

Server Monitoring

As I’m using Digital Ocean, monitoring server performance is a relatively simple process thanks to the built-in monitoring tools. For those not hosting with Digital Ocean, Netdata is a great alternative.

If you enabled monitoring during the Droplet creation, then you’re good to go! Otherwise, the following command will install the required agent.

curl -sSL https://agent.digitalocean.com/install.sh | sh

Once installed, you should see the Droplet’s current resource utilization when visiting the Droplet dashboard.

Monitoring dashboard

Alert Policies

Alert policies are extremely useful, and you should configure them for all Droplets. Alerts reduce the need to manually check your server’s health by sending a notification when one of your chosen metrics reaches a certain threshold.

Head to the Monitoring tab and click Create alert policy.

Create alert policies

Here, you can create as many alert policies as required. I like to set up alerts for CPU, memory and disk utilization with a threshold of 80%, which is a good starting point for most people.

Set up alert policies

Alert policies list

That’s all there is to server monitoring!

Initial Benchmarks

Although it isn’t necessary for you to perform this step, I want to show you how this setup handles traffic prior to any performance optimizations. It’s difficult to simulate web traffic, however, it is possible to send a large amount of concurrent requests to a server and track the responses. This gives you a rough indication of the amount of traffic a server can handle, but also allows you to measure the performance gains once you’ve implemented the optimizations.

The server I have setup for this series is a 512MB Digital Ocean Droplet. All of the tests in this post will be performed using Blitz, which is a cloud based performance and loading testing tool. Blitz makes it super easy to send a large amount of virtual users to your server in incremental stages and from various geographic regions. It also provides you with useful analytics, which can help to determine where bottlenecks are occurring. As of 2018, Blitz is no longer available. I’ve since switched to using ApacheBench, which is a CLI tool designed by the Apache Software Foundation.

The test I will perform will send an incremental amount of concurrent users to the server within a 60 second time period. The users scale, starting with 1 concurrent user and increasing to 200 concurrent users by the end of the test.

Here goes…

Full results here.

Based on the results the server can theoretically handle 2,885,760 requests a day with an average response time of 411. However, issues arose at around 142 concurrent users, which resulted in over 44% of visitors within the 60 second time period receiving a timeout or error. Not good at all!

With that out of the way, it’s time to optimize!

Object Cache

An object cache stores potentially computationally expensive data such as database query results and serves them from memory. This greatly improves the performance of WordPress as there is no longer a need to query the database on every page load for information already stored within the object cache.

Redis is the latest and greatest when it comes to object caching. However, popular alternatives include Memcache and Memcached.

To install Redis, issue the following commands.

sudo apt-get install redis-server

It’s also a good idea to set a maximum memory usage. As I’m only using a 512Mb server, I set mine to 64mb.

sudo nano /etc/redis/redis.conf

Uncomment the line # maxmemory and set the desired value.

maxmemory 64mb

Save the configuration and restart both Redis and PHP-FPM.

sudo service redis-server restart
sudo service php7.2-fpm restart

In order for WordPress to make use of Redis as an object cache you need to install the Redis Object Cache plugin by Till Krüss.

Object Cache - Plugins Screen

Once installed and activated, go to Tools > Redis to enable the object cache.

Object Cache - Enable

This is also the screen where you can flush the cache if required.

Object Cache - Flush

I’m not going to run the benchmarks again as the results won’t dramatically change. Although object caching reduces the average amount of database queries on the front page from 22 to 2, the database server is still being hit. Establishing a database connection on every page request is one of the biggest bottlenecks within WordPress.

The benefit of object caching can be seen when you look at the average database query time, which has decreased from 2.1ms to 0.3ms. The average query times were measured using Query Monitor.

If you check New Relic you’ll see how the server handles under heavy load. The spikes correspond to each test performed using Blitz. Clicking onto the processes shows that PHP is causing the high CPU usage.


New Relic

In order to further improve performance and decrease server resource usage you need to bypass PHP altogether. Enter page caching…

Page Cache

Although an object cache can go a long way to improving your WordPress site’s performance, there is still a lot of unnecessary overhead in serving a page request. For many sites, content is rarely updated. It’s therefore inefficient to load WordPress, query the database and build the desired page on every single request. Instead, you should serve a static HTML version of the requested page.

Nginx allows you to automatically cache a static HTML version of a page using the FastCGI module. Any subsequent calls to the requested page will receive the cached HTML version without ever hitting PHP.

Setup requires a few changes to your Nginx server block, so open your virtual host file.

sudo nano /etc/nginx/sites-available/ashleyrich.com

Add the following line before the server block, ensuring that you change the fastcgi_cache_path directive and keys_zone. You’ll notice that I store my cache within the site’s directory, on the same level as the logs and public directories.

fastcgi_cache_path /home/ashley/ashleyrich.com/cache levels=1:2 keys_zone=ashleyrich.com:100m inactive=60m;

You need to instruct Nginx not to cache certain pages. The following will ensure admin screens and pages for logged in users are not cached, plus a few others. This should go above the first location block.

set $skip_cache 0;

# POST requests and urls with a query string should always go to PHP
if ($request_method = POST) {
    set $skip_cache 1;
}   
if ($query_string != "") {
    set $skip_cache 1;
}   

# Don’t cache uris containing the following segments
if ($request_uri ~* "/wp-admin/|/xmlrpc.php|wp-.*.php|/feed/|index.php|sitemap(_index)?.xml") {
    set $skip_cache 1;
}   

# Don’t use the cache for logged in users or recent commenters
if ($http_cookie ~* "comment_author|wordpress_[a-f0-9]+|wp-postpass|wordpress_no_cache|wordpress_logged_in") {
    set $skip_cache 1;
}

Next, within the PHP location block add the following directives.

fastcgi_cache_bypass $skip_cache;
fastcgi_no_cache $skip_cache;
fastcgi_cache ashleyrich.com;
fastcgi_cache_valid 60m;

Notice how the fastcgi_cache directive matches the keys_zone set before the server block. In addition to changing the cache location, you can also specify the cache duration by replacing 60m with the desired duration in minutes. The default of 60 minutes is a good starting point for most people. Once happy, save the configuration.

Next you need to add the following directives to your nginx.conf file. The first instructs the FastCGI module on how to generate key names and the second adds an extra header to server responses so that you can easily determine whether a request is being served from the cache.

sudo nano /etc/nginx/nginx.conf

Add the following below the Gzip settings.

##
# Cache Settings
##

fastcgi_cache_key "$scheme$request_method$host$request_uri";
add_header Fastcgi-Cache $upstream_cache_status;

Save the configuration and restart Nginx.

sudo service nginx restart

Now when you visit the site and view the headers, you should see an extra parameter.

Nginx - Response Headers

The possible return values are:

  • HIT – Page cached
  • MISS – Page not cached (refreshing should cause a HIT or BYPASS)
  • BYPASS – Page cached but not served (admin screens or when logged in)

The final step is to install the Nginx Cache plugin, again by Till Krüss. This will automatically purge the FastCGI cache whenever your WordPress content changes. You can also manually purge the cache from the WordPress dashboard.

Once installed, navigate to Tools > Nginx and define your cache zone path. This should match the value you specified in your Nginx hosts file.

Final Benchmarks

With the optimizations out of the way it’s time to perform a final benchmark. This time I’m going to up the maximum concurrent users to 1,000.

Full results here.

Not bad at all! The average response time has dropped to 44ms, with a new theoretical limit of 40,632,480 requests per day, which is pretty phenomenal for a $5/mo server.

Now when you take a look at New Relic you will notice that the CPU spike is much less severe. You’ll also notice that PHP is no longer causing the spike as the heavy lifting is handled by Nginx.


New Relic

Performance optimization is a lot more difficult on highly dynamic sites where the content updates frequently, such as those that use bbPress or BuddyPress. In these situations it’s often required to disable page caching on the dynamic sections of the site (the forums for example). This is achieved by adding additional rules to the skip cache section within the Nginx server block. This will force those requests to always hit PHP and generate the page on the fly. Doing so will often mean you have to scale hardware sooner, thus increasing server costs.

Caching Plugins

At this point you may be wondering why I chose this route instead of installing a plugin such as W3 Total Cache or WP Super Cache. Firstly, not all plugins include an object cache and for those that do you will often need to install additional software on the server (Redis for example) in order to take full advantage of the feature. Secondly, the static pages generated by these plugins are often more computationally expensive to generate as the processing is done by PHP. Using Nginx and the FastCGI module will often lower CPU and memory usage, which is ideal on servers with less system resources. Thirdly, have you seen the settings screens within these plugins recently!?

Finally, if you prefer to run WordPress with fewer plugins you can actually replace the Redis Object Cache plugin with a drop-in file. You can also remove the Nginx Cache plugin, however, the page cache will no longer conditionally purge when new posts or comments are published. This isn’t however a problem for completely static sites, as you can always delete the contents of the cache folder (which you created earlier) if you need to flush the cache.

That concludes this post. Next time I’ll guide you through the process of handling outgoing email, configuring cron and scheduling automatic backups. Until then!

About the Author

Ashley Rich

Ashley is a PHP and JavaScript developer with a fondness for hosting, server performance and security. Before joining Delicious Brains, Ashley served in the Royal Air Force as an ICT Technician.