Functional tests are crucial for WP-CLI packages as they simulate real-world scenarios, test integration with WordPress, and how they behave in a command line environment. In this article, I’ll show you how to write functional tests for WP-CLI packages.

What Are Functional Tests?

You may be familiar with unit testing using tools like PHPUnit. Unit tests are designed to test individual units of code, such as functions or methods, in isolation. The goal of unit testing is to verify that each unit of code works correctly on its own, without any dependencies on other parts of the system. Unit tests are a great way to test specific functions or methods in your projects, and focus more on the code side of things.

Functional tests, on the other hand, are designed to test the system as a whole, with all of its dependencies and interactions with other systems. Instead of writing code to verify that your existing code works properly, you describe how you expect a specific feature to behave in human-readable terms. This can be done using a language called Gherkin, for example.

This is especially helpful for WP-CLI, because it allows you to confirm that your code is working properly, and that any output displayed to the user matches with what is expected.

Writing Functional Tests

Getting Started with Behat

WP-CLI uses Behat to run its functional tests, so in this case that’s what we’re going to do as well. If you’re not already familiar with Behat, it may be worth skimming the official documentation to get a basic sense of what it is and how it works.

If you’d like to following along with writing the tests in this article, you can clone down the WP Installer Git repository using [email protected]:mattgrshaw/wp-installer.git. Alternatively, you can follow the same basic steps for any other WP-CLI package that you’d like to write tests for.

Next, if you don’t have it installed already, install the wp scaffold package command by running wp package install wp-cli/scaffold-package-command. You can use this command to generate the files and dependencies needed to test the package by running wp scaffold package tests. Finally, run bin/install-package-tests.sh to set up the testing environment.

Once that’s done, you can run ./vendor/bin/behat in the project directory to confirm that everything is working:

As you can see, a sample test was included when we scaffolded the package tests. This is located at features/load-wp-cli.feature, and is worth a look if you’re not familiar with writing functional tests in Gherkin. Otherwise, you can delete that file and get started on writing your own tests.

Writing Some Tests

Getting to the good stuff, it’s very easy to write our own tests since they are mostly written in plain English. For example, let’s create a basic test that verifies that the package is activated and the wp help installer command is working as intended. This can be done by creating the file features/load-wp-installer.feature with the following contents:

Feature: Test that WP Installer loads.

  Scenario: WP Installer loads correctly
    Given a WP install
    When I run `wp help installer`
    Then STDOUT should contain:
      """
      wp installer <command>
      """

In this test, we tell Behat to run wp help installer, and make sure that the output of the command contains a string that should only exist if the WP-CLI package loaded correctly. Behat picks up on certain keywords and phrases like “Given”, “When”, and “Then”. But where does all this come from? How does it know what a “WP install” is and what it should do with it?

If you look in features/steps, you’ll see three files: given.php, then.php, and when.php. Each of these files contains the information necessary to use the keyword. Here’s a look at some of the steps defined in features/steps/given.php:

$steps->Given( '/^wp-config\.php$/',
    function ( $world ) {
        $world->create_config();
    }
);

$steps->Given( '/^a database$/',
    function ( $world ) {
        $world->create_db();
    }
);

$steps->Given( '/^a WP install$/',
    function ( $world ) {
        $world->install_wp();
    }
);

Each step calls the method Given(), which contains a RegEx expression for the first parameter, and a function to be called if that expression is matched. The same format applies for the steps defined in features/steps/then.php and features/steps/when.php.

Let’s try writing another test to get the hang of things. The wp installer install command is responsible for creating new WordPress installs, so it’d be good to add some test coverage to make sure that installs are getting created and the command behaves as expected.

To do this, create a new file at features/installer-install.feature:

Feature: Test the `wp installer install` command.

  Scenario: Install WordPress
    When I run `wp installer install wptest --site_base_path=/tmp/ --dbuser=root --dbpass=root --dbhost=127.0.0.1`
    Then STDOUT should contain:
      """
      Downloading WordPress...
      Creating wp-config.php...
      Creating the database...
      """
    And the /tmp/wptest directory should contain:
      """
      index.php
      license.txt
      readme.html
      wp-activate.php
      wp-admin
      wp-blog-header.php
      wp-comments-post.php
      wp-config-sample.php
      wp-config.php
      wp-content
      wp-cron.php
      wp-includes
      wp-links-opml.php
      wp-load.php
      wp-login.php
      wp-mail.php
      wp-settings.php
      wp-signup.php
      wp-trackback.php
      xmlrpc.php
      """

The above test verifies that the command output matches up with what we’re expecting, and makes use of some additional steps to verify the WordPress files are present in the directory, and the wp-config.php file was created and configured as needed.

It’s definitely helpful to be able to test the UI (output) and the functionality of the command in the same test, even when there’s a lot going on under the hood of the command being tested.

Creating Our Own Step

So far we’ve been using the steps for Given, Then, and And that were already provided for us when scaffolding the package tests. But what if we need to add our own custom step to check something that hasn’t been covered yet?

In this case, we already know that the WordPress files have been created and the wp-config.php file was generated. But the tests don’t check if the database was created for the install. We can easily add that check by adding a new step in features/steps/then.php:

$steps->Then( '/^the database for the install at (.+) should (exist|not exist)$/',
    function( $world, $path, $action ) {
        $proc   = Process::create( 'wp db tables', $path );
        $result = $proc->run();


        if ( 'exist' === $action && 1 === $result->return_code ) {
            throw new Exception( $world->result );
        } elseif ( 'not exist' === $action && 0 === $result->return_code ) {
            throw new Exception( $world->result );
        }
    }
);

The RegEx in the first part of the step checks the path of the WordPress install and whether the database should or shouldn’t exist, and passes that information to the function defined in the second parameter. That function then launches a process that runs wp db tables in the provided path, which confirms that the database is installed and contains the WordPress database tables.

We can now use this step to check to test if the database is created successfully using the wp installer install command. Add the following line to the scenario tested in installer-install.feature:

And the database for the install at /tmp/wptest should exist

Now when we run ./vendor/bin/behat. We should see our updated tests, which should pass if the files were installed and the database was created successfully.

Next Steps

Functional testing really lends itself well to behavior-driven development since it helps you stop and think about what the command should look like and what it should do before writing any code. Also, getting in the habit of writing any tests can undoubtedly help you as both a developer and as a project maintainer.

Have you written functional tests before (using Gherkin/Behat or another tool)? Is it something you’re likely to do in the future? Let me know in the comments below.