We’ve written before about how we are using automated acceptance testing to test our WordPress plugins which has decreased manual testing time and helped catch bugs before they are released.

But it wasn’t until recently that we applied this same standard to our site itself.

Last year I was tasked with looking after deliciousbrains.com, our WooCommerce site where we sell our plugins (the same site you’re on now reading this). Which is kind of a big responsibility. If I deploy a bug it could stop people from purchasing and they might not return! My mistakes can directly and immediately impact our bottom line. 😬

We have always had manual testing in place on the site. In fact, I wrote about our comprehensive approach to upgrading WooCommerce before, which involved thorough testing of our ‘critical path’:

• Purchasing a plugin with PayPal
• Purchasing a plugin with Stripe
• Upgrading a license
• Testing an auto renewal.

But performing these tests manually every time I make a code change or update some plugins can get tedious, especially now that I know the ease of automated testing.

So a while back I spent some time setting up the site with Codeception, our automated testing tool of choice, and recreated all of our manual tests. What follows is the approach I took, along with some tips and tricks to get you up-and-running testing a WordPress site running WooCommerce.

This guide could also apply to any WordPress site – ecommerce, membership or otherwise – where you’re concerned about a critical path and want to automate your testing process.

Getting Started

Before you start it’s a good idea to plan out the testing scenarios you want to write, based on the critical path or manual tests you already perform. Think about who will be performing the test, eg. a customer or administrator on your site and what attributes they have (logged in, returning customer, etc.). You’ll see these ‘actors’ covered more later.

First we need to install and configure Codeception:

composer require codeception/codeception --dev
vendor/bin/codecept bootstrap

When testing WordPress sites with Codeception, the most important module to install is Luca Tume’s WPBrowser:

composer require lucatume/wp-browser --dev

WPBrowser requires you to have a working WordPress site for it to access and perform the tests on. As I’m only running the tests locally right now, I’m using the same development site I use for development and manual testing.

WPBrowser requires some extra configuration, kicked off with the command:

vendor/bin/codecept init wpbrowser

WPBrowser has a number of modules to help testing WordPress, but they don’t all need to be used. I ended up only using the WPDb and WPWebDriver modules (with Chromedriver) for my tests.

WPDb extends the default Codeception Db module, and has specific knowledge of the WordPress database structure and tables.

WPWebDriver is a Guzzle-based and JavaScript capable web driver that should be used in conjunction with a Selenium server, PhantomJS, or any real web browser. For our purposes, I’ve opted for using browser testing instead of a headless browser, as the tests will need to interact with Javascript, and it’s much easier to follow what’s happening when debugging tests.

I configured the modules in my acceptance.suite.yml config file as follows:

actor: AcceptanceTester
modules:
    enabled:
        - WPDb
        - WPWebDriver
    config:
        WPDb:
            dsn: 'mysql:host=%DB_HOST%;port=%DB_PORT%;dbname=%DB_NAME%'
            user: '%DB_USER%'
            password: '%DB_PASSWORD%'
            populate: false #import the dump before the tests
            cleanup: false #import the dump between tests
            url: '%WP_URL%'
            urlReplacement: true #replace the hardcoded dump URL with the one above
            tablePrefix: '%TABLE_PREFIX%'
        WPWebDriver:
            url: '%WP_URL%'
            browser: chrome
            window_size: false # disabled in ChromeDriver
            port: 9515
            adminUsername: '%ADMIN_USERNAME%'
            adminPassword: '%ADMIN_PASSWORD%'
            adminUrl: /wp/wp-admin

Writing Tests

My first test is the purchase of a license with PayPal. You can quickly generate a test file with:

codecept generate:cept acceptance 00-001-PayPalOrder-Cept.php

The first thing I want my test to perform is to navigate to the pricing page for WP Migrate DB Pro and then click ‘Buy Now’. The plugin license will be added to the cart and the site should redirect to the checkout screen.

This is the code I would write in my Cept file. Codeception has two test file styles; cept is a procedural style, whereas cest is in class format. I am checking at the end that the correct product is displayed as the order summary on the checkout screen:

$I = new AcceptanceTester( $scenario );
$I->wantTo('add a plugin to the cart and check it gets added');
$I->amOnPage( '/wp-migrate-db-pro/pricing' );
$I->waitForText( 'DEVELOPER' );
$I->click( '.dbi-pricing__plan--migrate-db a.dbi-btn' );
$I->waitForText( 'Order Summary', 100 );
$I->see( 'WP Migrate DB Pro' );
$I->see( 'Developer' );

It’s just a matter of writing code to simulate the clicking and waiting through the purchase screen until we have purchased the plugin:

• Login to PayPal Sandbox with a test user
• Checkout with PayPal
• Ensure we return to the site with the order confirmation screen displayed
• Test downloading of the file zip
• Test order confirmation email received

The last two steps involved some custom modules that I’ll come onto later.

Preparing

The nature of writing these tests is that you will run, tweak the steps, and run them again – a lot! This means you want the data to be in the right state before tests are run, not left in a half-baked state after previous tests.

I started by adding this to the top of my testcase, so my customer was always logged out and then the user deleted from the database.

$I = new AcceptanceTester( $scenario );
$I->logOut();
$I->dontHaveUserInDatabaseWithEmail( $email );

I realized I would need to call this at the start of every test so I began to look for a better way to make the code reusable. Turns out Codeception makes this really easy.

Extending Codeception

As you will notice from my test code, the first line creates a new instance of the AcceptanceTester class. This is stored as a variable $I, so it makes more sense when using the natural language of the test code. For example, I log out, I click this.

The AcceptanceTester class extends from the Codeception ‘Actor’ class. The actor can perform all actions and assertions from the modules that are defined in the acceptance.suite.yml file which I looked at earlier.

You can add widely-used code to the AcceptanceTester class, which is available to extend inside the file tests/_support/Step/AcceptanceTester.php. For example, if I want my testing actor to easily navigate to the ‘My Account’ page on our site. WPBrowser adds a handy amOnPage method to open a page for a given relative URI, relative to the WordPress site home url. I can use that in my method:

public function amOnAccount( $page = '' ) {
    $this->amOnPage( '/my-account/' . $page );
    $this->waitForText( 'My Account' );
}

As part of my cleanup before tests are run, I want to delete the test user from the database, but I also want to delete any previous orders from the database, which involves deleting rows from the ‘wp_posts’, ‘wp_woocommerce_software_licences’, and ‘wp_woocommerce_software_subscriptions’ tables:

public function dontHaveOrdersInDatabaseForEmail( $userEmail ) {
    $licenses = $this->grabAllFromDatabase( $this->grabPrefixedTableNameFor( 'woocommerce_software_licences' ), '*', [ 'activation_email' => $userEmail ] );
    foreach ( $licenses as $license ) {
        $this->dontHavePostInDatabase( [ 'ID' => $license['order_id'] ] );
        $this->dontHaveInDatabase( $this->grabPrefixedTableNameFor( 'woocommerce_software_licences' ), [ 'key_id' => $license['key_id'] ] );
        $this->dontHaveInDatabase( $this->grabPrefixedTableNameFor( 'woocommerce_software_subscriptions' ), [ 'key_id' => $license['key_id'] ] );
    }
}

Custom Actors

There is some functionality that won’t be shared by all the actions an actor might perform. For example, I want to test a customer buying a plugin, but I also want to test an administrator refunding a purchase. These are two different types of actors with different types of actions to be performed. Codeception allows you to create new actors in the form of StepObjects.

You can quickly create a new class using the following command:

vendor/bin/codecept generate:stepobject acceptance Administrator

This will create a new file tests/_support/Step/Acceptance/Administrator.php, which I have then added my custom methods to:

namespace Step\Acceptance;

use Codeception\Scenario;

class Administrator extends \AcceptanceTester {

    public function __construct( Scenario $scenario ) {
        parent::__construct( $scenario );
        $this->logOut();
        $this->loginAsAdmin();
    }

    public function refundOrder( $orderID ) {
        $this->amOnOrderPage( $orderID );
        $this->waitForText( "Order #$orderID details" );
        $this->click( 'Refund' );
        $this->wait(2);
        $this->fillField( '.refund_order_item_qty', 1 );
        $this->click( '.do-api-refund' );
        $this->acceptPopup();
        $this->waitForText( "Refunded" );
    }
}

In my tests where I want to perform a refund, the order ID will typically be the last order created in my test database. Fetching this ID from the database is something that the WPDb module has methods to help with. It actually has a method called grabLatestEntryByFromDatabase which accepts a table and column name arguments. However, I want to get the last ID from the posts table for a specific post_type, in this case shop_order.

Luckily I can make my own version of the WPDb module, extend it and add my method. Codeception allows you to create ‘Helper’ classes that sit inside tests/_support/Helper/.

Here’s my Helper for WPDb, where I’ve used the similar code found in grabLatestEntryByFromDatabase to run my specific query:

namespace Helper;

class WPDb extends \Codeception\Module\WPDb {
    public function grabLastOrderID() {
        $idColumn  = 'ID';
        $tableName = $this->grabPostsTableName();
        $postType  = 'shop_order';

        $dbh = $this->_getDbh();
        $sth = $dbh->prepare( "SELECT {$idColumn} FROM {$tableName} WHERE post_type = '{$postType}' ORDER BY {$idColumn} DESC LIMIT 1" );
        $this->debugSection( 'Query', $sth->queryString );
        $sth->execute();
            
        return $sth->fetchColumn();
    }
}

To get Codeception to load my WPDb module, I need to tweak my acceptance.suite.yml:

actor: AcceptanceTester
    modules:
        enabled:
-           - WPDb
+          - \Helper\WPDb

Considerations

There are a few things to consider when testing an ecommerce site like the WooCommerce deliciousbrains.com. I want to test as much as possible which means using extra modules and writing code to get around some issues with third parties.

Payment Gateways

We use two payment gateways on the site: PayPal and Stripe. As you would expect, PayPal has been the more difficult one to test 😂

We already have code that ensures we use the test and sandbox sites for both payment gateways, but for PayPal you will need to have a sandbox email address and password to use for the test payments. The test suite .env file requires these credentials:

PAYPAL_SANDBOX_USERNAME=”"
PAYPAL_SANDBOX_PASSWORD=""

They can then be referenced like $_ENV['PAYPAL_SANDBOX_USERNAME'] in my test code. Stripe has test card details that can be used.

I had some fun and games traversing the PayPal login and checkout screens using Codeception’s web browser (Selenium-based), but eventually got the checkout working, here abstracted to a method in my ‘Customer’ actor stepObject class:

public function checkoutPayPal() {
    $I = $this;
    $I->click( '.dbi-use-paypal a' );
    // Login to PayPal Sandbox
    $I->waitForElementNotVisible('#preloaderSpinner', 100 );
    $I->waitForText('PayPal', 100 );
    $I->waitForElementVisible( '#email', 100 );
    $I->fillField( 'login_email', $_ENV['PAYPAL_SANDBOX_USERNAME'] );
    if ( $I->seeOnPage( 'Next' ) ) {
        $I->click( 'Next' );
        $I->waitForElementVisible( '#password', 100 );
    }
    $I->fillField( 'login_password', $_ENV['PAYPAL_SANDBOX_PASSWORD'] );
    $I->click( 'Log In' );
    $I->waitForElementNotVisible('#preloaderSpinner', 100 );
    if ( $I->seeOnPage( 'Continue' ) && ! $I->seeOnPage( 'Agree and Continue' ) ) {
            $I->click( 'Continue' );
        }
    $I->wait(5 );
    $I->waitForText( 'Information from Delicious Brains', 100 );
    // Confirm PayPal agreement
    $I->click( '#confirmButtonTop' );
}

There’s an extra conditional to check if PayPal is asking a question before it gets to the final submit for the checkout. This came about a few months ago and broke the existing tests, so I had to add a custom method to my AcceptanceTester class which allows me to test if there is text on the page without bombing the tests if it doesn’t exist:

/**
* Check if text exists on page to be used in a conditional.
* 
* @param string $text
*
* @return bool
*/
public function seeOnPage( $text ) {
    try {
        $this->see( $text );
    } catch ( \PHPUnit\Framework\ExpectationFailedException $f ) {
        return false;
    }

    return true;
}

Emails

When new orders are created, the site sends out a couple of emails to the customer informing them of the order details and their new account. I want to test that these emails get sent and contain the correct information, such as the plugin name of what was purchased.

When developing locally we use MailTrap to ensure no emails get sent. We use a local mu-plugin to make WordPress use MailTrap for email sending, but getting the site to do this when running the tests for any developer, meant I needed a bit of extra configuration.

I created a PHP file which will serve as an mu-plugin for the site when the tests are running, which lives in tests/_data/mu-plugins:

if ( ! defined( 'DBRAINS_HIJACK_ALL_MAIL' ) ) {
    define( 'DBRAINS_HIJACK_ALL_MAIL', '[TEST_EMAIL]' );
}

if ( ! defined( 'DBI_AUTO_RENEWAL_SUBSCRIPTION_LIMIT' ) ) {
    define( 'DBI_AUTO_RENEWAL_SUBSCRIPTION_LIMIT', 1 );
}
if ( ! defined( 'DBI_AUTO_EMAIL_LIMIT' ) ) {
    define( 'DBI_AUTO_EMAIL_LIMIT', 1 );
}

add_filter( 'option_active_plugins', function ( $plugins ) {
    if ( ! is_array( $plugins ) || empty( $plugins ) ) {
        return $plugins;
    }
    foreach ( $plugins as $key => $plugin ) {
        if ( 'mailgun/mailgun.php' === $plugin ) {
            unset( $plugins[ $key ] );
        }
    }

    return $plugins;
} );

add_action( 'phpmailer_init', function ( $phpmailer ) {
    $phpmailer->isSMTP();
    $phpmailer->Host     = 'smtp.mailtrap.io';
    $phpmailer->SMTPAuth = true;
    $phpmailer->Port     = 2525;
    $phpmailer->Username = '[TEST_MAILTRAP_USERNAME]';
    $phpmailer->Password = '[TEST_MAILTRAP_PASSWORD]';
} );

There are a few pieces of data that need to be replaced with the actual testing data at runtime, and this mu-plugin needs to be in place on the WordPress site. This is handled by some code in the Acceptance.php custom module in tests/_support/Helper:

namespace Helper;

// here you can define custom actions
// all public methods declared in helper class will be available in $I

class Acceptance extends \Codeception\Module {

    protected $mu_plugin = 'acceptance-testing.php';

    public function _beforeSuite( $settings = [] ) {
        parent::_beforeSuite();

        $email_plugin_contents = file_get_contents( codecept_data_dir() . 'mu-plugins/' . $this->mu_plugin );

        $email_plugin_contents = str_replace( '[TEST_EMAIL]', $_ENV['ADMIN_EMAIL'], $email_plugin_contents );
        $email_plugin_contents = str_replace( '[TEST_MAILTRAP_USERNAME]', $_ENV['MAILTRAP_USERNAME'], $email_plugin_contents );
        $email_plugin_contents = str_replace( '[TEST_MAILTRAP_PASSWORD]', $_ENV['MAILTRAP_PASSWORD'], $email_plugin_contents );

        $this->deleteEmailPlugin();
        file_put_contents( dirname( codecept_root_dir() ) . '/public_html/content/mu-plugins/' . $this->mu_plugin, $email_plugin_contents );
    }

    public function _afterSuite() {
        parent::_afterSuite();

        $this->deleteEmailPlugin();
    }

    protected function deleteEmailPlugin() {
        $email_plugin = dirname( codecept_root_dir() ) . '/public_html/content/mu-plugins/' . $this->mu_plugin;
        if ( file_exists( $email_plugin ) ) {
            unlink( $email_plugin );
        }
    }
}

When it comes to actually testing emails, there is a third-party Codeception module for MailTrap which allows you to look in your MailTrap inbox and make assertions about emails. For example, these lines wait for an email with the specified text in the body to arrive then checks for text in the subject and body:

$I->waitForEmailWithTextInHTMLBody( 'Order Complete', 100 );
$I->seeInEmailSubject( 'Order Complete' );
$I->seeInEmailHtmlBody( ‘WP Migrate DB Pro - Developer' );

I plan to extend this module in the future so I can test emails sent against a saved copy to ensure all the style and formatting is correct, as well as testing all the links work correctly.

Running Tests

In order for these tests to run quickly and easily, I use a bash file tests/bin/run-acceptancetests.sh which runs the Codeception build command, kills any existing Chromedriver process, launches Chromedriver and then finally runs the acceptance suite of tests.

#!/usr/bin/env bash

REPO_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )/../.." && pwd )"
export PATH="${REPO_DIR}/vendor/bin:${PATH}"
cd "${REPO_DIR}/tests"

codecept build

kill -9 $(pgrep chromedriver)
chromedriver --url-base=/wd/hub &

codecept run acceptance "$@"

kill -9 $(pgrep chromedriver)

I would like to eventually get these tests being run by Travis as part of the lifecycle of pull requests, but there will most likely be some configuration challenges which could make for another post.

What’s Next

Having this suite of tests has given me peace of mind when developing the site, and more importantly, it’s made my life easier! I don’t have to perform manual, boring, testing all the time.

I can also add more tests later to try and fully cover a lot of our important features of the site, like our licensing API and the interactions with our email marketing service.

Do you perform automated testing on your site? What tools do you use? Let us know in the comments.