Over the holidays in 2022, I refactored a lot of the code for Secret Santa Hat. After seeing hundreds of new groups being created each day, I realized it was no longer a simple pet project. No longer could I just push development code directly to production. The project needed to mature.

So, I added some QA processes to ensure that anything deployed live is functional. This gave me the opportunity to really deep-dive end-to-end(E2E) tests and put them in place. I’ve learned a lot, and I see the huge value this can add to my work as a WordPress developer by adding E2E tests to the plugins I build for clients.

What are E2E Tests?

End-to-End(E2E) tests are automated tests that validate your code via a web browser instance. In essence, a “worker” is spawned that opens a browser context and performs a set of actions that mimic a real user. The results of these actions can then be validated against the expected results.

Automated actions include navigating links, filling forms, logging in or out, scrolling, and a whole lot more. Pretty much anything a real user can do to your app/website, an E2E worker can as well.

Why?

To prevent deploying bad code to production! In my case, I started really small. I initially only validated that the home page loaded. This simple E2E test at least confirmed the entire site would not be broken by a change I wanted to make. From there, the tests grew both in number and complexity.

For Secret Santa Hat, I wrote tests that validated the basic site navigation, creating a Secret Santa group with various names, adding phone numbers, drawing names from the hat, updating a user’s wishlist, and more.

WordPress Plugins

When I added my first E2E test for some WordPress plugin code, I again started small. I had the test open the default login page and validate that the “Powered by WordPress” heading existed somewhere in the DOM. This confirmed there were no major, site-crippling fatal errors as a result of the plugin’s code.

Following a similar pattern, these tests have grown in number and complexity. I’ve learned a whole lot, so I figured it’s time for a tutorial on how to get started with E2E tests for WP plugins and a sample of some simple tests.

Getting Started with Playwright

I’ve been using Playwright for E2E testing. There are other solutions out there, but I’ve found that Playwright has good documentation. Also, there’s an effort in WordPress Core and the Gutenberg Editor to migrate their E2E tests from Puppeteer to Playwright. As this effort continues, no doubt there will be a lot more fixtures and scaffolding made available to plugin and theme developers to use in their own projects.

Installing Playwright

Installing Playwright is pretty simple. Navigate to your plugin or theme’s root directory and run:

npm init playwright@latest

This will install the dependencies, create a starter playwright.config.js configuration file, install the browsers, and add some sample tests. If you have trouble installing, refer to the docs to install manually.

During installation, you’ll be asked to choose to use either TypeScript or JavaScript as the project language. From here onward, I’ll just reference everything as if this were a JavaScript project.

And that should be it. Playwright is now installed as a dependency in your project directory. Let’s work on some tests now.

Organization

If you’re using PHPUnit for unit testing of your code, you probably already have a tests/ directory. In that case, I suggest moving the E2E tests to a subdirectory, e.g. tests/e2e/. This was my setup as well, so the base directory for E2E tests for the remainder of this post will reference that path.

Going a step further, grouping the tests by purpose would be beneficial. For instance, if you’re testing some specific functionality related to front end comments, you might group them under tests/e2e/front-end-comments/.

Finally, Playwright runs tests in alphanumeric order. Once trick I saw to run my tests in a specific order was to prefix them with numbers. So, groups of tests might be under tests/e2e/001-front-end-comments/ and sequential tests inside this group might include tests/e2e/001-front-end-comments/001-first-test.spec.js and tests/e2e/001-front-end-comments/002-second-test.spec.js.

Global Setup

In order to bootstrap each set of tests, we can add a globalSetup.js file to the tests/e2e/ directory. This file allows us to fire up a browser session and start with a base URL. A very basic tests/e2e/globalSetup.js might look like this:

This code reuses the config passed in from playwright.config.js, so we’ll need to update that as well.

The main thing to change here is the baseURL. This should be the local development URL. There’s a LOT more that can be configured in this file, so be sure to have a look at the docs for more information.

Writing a Basic Test

First, in the tests/ directory, let’s rename the tests/example.spec.js file to tests/basic-login.spec.js. We’ll use this test file only to validate that the local development copy of the site is not experiencing a site-wide fatal error.

Let’s open tests/basic-login.spec.js and start with this basic test:

So, what does this code do? Each test is run by a worker, and it’s given a page instance that comes from the globalSetup. With this page instance, we can do all sorts of things.

In this test, we’re instructing the browser to navigate to the default WordPress login page. Since we’re awaiting the page navigation, the next bit of code will not run until that page is loaded. The length of the wait timeout can be adjusted in the main playwright.config.js file or given as options to some methods.

Once the page is loaded, we’re using the getByRole “Locator” method to find a heading text on the page, specifically “Powered by WordPress”. Locators are half the magic of Playwright, and they allow you to find all sorts of elements and properties of pages. Be sure to look at all the Locators available to you.

Now, the tests needs to validate that the H1 contains the text “Powered by WordPress” and that it exists in the DOM. To do that, we use the toBeVisible Assertion. Assertions are the other half of the magic of Playwright. These methods can validate all sorts of states of elements or properties on the page. There are a lot of different Assertions, so check them out in the docs.

Again, we’re awaiting the locator assertion, giving it time to appear in the DOM. The default timeout is set in the playwright.config.js config, but you can also pass a custom timeout in the assertion’s option object.

Running The Tests

Now that we have written our first basic test, we can run it. From the project’s root directory, run npx playwright test. You can also add it as a script in your package.json. This allows you to customize the script. For example, if you have a build process, you might have a script like this:

Running npm run test:e2e will now create a production bundle of your project code before running the E2E tests. There’s several options you can pass to npx playwright test so be sure to check out the docs.

Once you run the tests, it will report successes and failures. For our first test, it should be a success. If not, run npx playwright test --debug. This will actually run Playwright in “headed” mode, opening a browser window for you to see on each test file. This will allow you to step through tests and see what’s actually going on on the page while the tests are running.

Tests run really fast, so if you need to bake some waiting time into a test for you to investigate the browser window, you can add something like page.waitForTimeout(5000) which causes the runner to pause actions for 5 seconds.

What’s Next?

First, I would make this part of your plugin development workflow. Use git hooks to run the E2E testing when appropriate, e.g. precommit. You can use something like Husky to easily manage the hooks.

You can also integrate the tests into your existing CI to run when a new release is created.

At this point, we have _some_ peace of mind that our plugin code doesn’t break the entire site. However, there’s a lot more we can do. We can start adding tests that validate our specific plugin’s functionality. Tests to check that logging in works fine, check that our plugin adds a menu item to the admin, check that our plugin’s settings page loads and saves settings properly, etc. Don’t forget “unhappy path” tests!

You can even validate functionality on specific browsers and specific viewport sizes. So, you can add tests that validate your mobile menu functionality separate from the desktop navigation.

In a future post, we’ll work with reusable fixtures, which are invaluable helpers when performing the same task or requiring a similar state across multiple tests.

About David

Web Developer