Over the holidays in 2022, I refactored a lot of the code for Secret Santa Hat. After seeing hundreds of new groups being created each day, I realized it was no longer a simple pet project. No longer could I just push development code directly to production. The project needed to mature.
So, I added some QA processes to ensure that anything deployed live is functional. This gave me the opportunity to really deep-dive end-to-end(E2E) tests and put them in place. I’ve learned a lot, and I see the huge value this can add to my work as a WordPress developer by adding E2E tests to the plugins I build for clients.
What are E2E Tests?
End-to-End(E2E) tests are automated tests that validate your code via a web browser instance. In essence, a “worker” is spawned that opens a browser context and performs a set of actions that mimic a real user. The results of these actions can then be validated against the expected results.
Automated actions include navigating links, filling forms, logging in or out, scrolling, and a whole lot more. Pretty much anything a real user can do to your app/website, an E2E worker can as well.
To prevent deploying bad code to production! In my case, I started really small. I initially only validated that the home page loaded. This simple E2E test at least confirmed the entire site would not be broken by a change I wanted to make. From there, the tests grew both in number and complexity.
For Secret Santa Hat, I wrote tests that validated the basic site navigation, creating a Secret Santa group with various names, adding phone numbers, drawing names from the hat, updating a user’s wishlist, and more.
When I added my first E2E test for some WordPress plugin code, I again started small. I had the test open the default login page and validate that the “Powered by WordPress” heading existed somewhere in the DOM. This confirmed there were no major, site-crippling fatal errors as a result of the plugin’s code.
Following a similar pattern, these tests have grown in number and complexity. I’ve learned a whole lot, so I figured it’s time for a tutorial on how to get started with E2E tests for WP plugins and a sample of some simple tests.
Getting Started with Playwright
I’ve been using Playwright for E2E testing. There are other solutions out there, but I’ve found that Playwright has good documentation. Also, there’s an effort in WordPress Core and the Gutenberg Editor to migrate their E2E tests from Puppeteer to Playwright. As this effort continues, no doubt there will be a lot more fixtures and scaffolding made available to plugin and theme developers to use in their own projects.
Installing Playwright is pretty simple. Navigate to your plugin or theme’s root directory and run:
npm init playwright@latest
This will install the dependencies, create a starter
playwright.config.js configuration file, install the browsers, and add some sample tests. If you have trouble installing, refer to the docs to install manually.
And that should be it. Playwright is now installed as a dependency in your project directory. Let’s work on some tests now.
If you’re using PHPUnit for unit testing of your code, you probably already have a
tests/ directory. In that case, I suggest moving the E2E tests to a subdirectory, e.g.
tests/e2e/. This was my setup as well, so the base directory for E2E tests for the remainder of this post will reference that path.
Going a step further, grouping the tests by purpose would be beneficial. For instance, if you’re testing some specific functionality related to front end comments, you might group them under
Finally, Playwright runs tests in alphanumeric order. Once trick I saw to run my tests in a specific order was to prefix them with numbers. So, groups of tests might be under
tests/e2e/001-front-end-comments/ and sequential tests inside this group might include
In order to bootstrap each set of tests, we can add a
globalSetup.js file to the
tests/e2e/ directory. This file allows us to fire up a browser session and start with a base URL. A very basic
tests/e2e/globalSetup.js might look like this:
This code reuses the
config passed in from
playwright.config.js, so we’ll need to update that as well.
The main thing to change here is the
baseURL. This should be the local development URL. There’s a LOT more that can be configured in this file, so be sure to have a look at the docs for more information.
Writing a Basic Test
First, in the
tests/ directory, let’s rename the
tests/example.spec.js file to
tests/basic-login.spec.js. We’ll use this test file only to validate that the local development copy of the site is not experiencing a site-wide fatal error.
tests/basic-login.spec.js and start with this basic test:
So, what does this code do? Each
test is run by a worker, and it’s given a
page instance that comes from the
globalSetup. With this
page instance, we can do all sorts of things.
In this test, we’re instructing the browser to navigate to the default WordPress login page. Since we’re
awaiting the page navigation, the next bit of code will not run until that page is loaded. The length of the wait timeout can be adjusted in the main
playwright.config.js file or given as options to some methods.
Once the page is loaded, we’re using the
getByRole “Locator” method to find a heading text on the page, specifically “Powered by WordPress”. Locators are half the magic of Playwright, and they allow you to find all sorts of elements and properties of pages. Be sure to look at all the Locators available to you.
Now, the tests needs to validate that the
H1 contains the text “Powered by WordPress” and that it exists in the DOM. To do that, we use the
toBeVisible Assertion. Assertions are the other half of the magic of Playwright. These methods can validate all sorts of states of elements or properties on the page. There are a lot of different Assertions, so check them out in the docs.
awaiting the locator assertion, giving it time to appear in the DOM. The default timeout is set in the
playwright.config.js config, but you can also pass a custom
timeout in the assertion’s
Running The Tests
Now that we have written our first basic test, we can run it. From the project’s root directory, run
npx playwright test. You can also add it as a script in your
package.json. This allows you to customize the script. For example, if you have a build process, you might have a script like this:
npm run test:e2e will now create a production bundle of your project code before running the E2E tests. There’s several options you can pass to
npx playwright test so be sure to check out the docs.
Once you run the tests, it will report successes and failures. For our first test, it should be a success. If not, run
npx playwright test --debug. This will actually run Playwright in “headed” mode, opening a browser window for you to see on each test file. This will allow you to step through tests and see what’s actually going on on the page while the tests are running.
Tests run really fast, so if you need to bake some waiting time into a test for you to investigate the browser window, you can add something like
page.waitForTimeout(5000) which causes the runner to pause actions for 5 seconds.
First, I would make this part of your plugin development workflow. Use git hooks to run the E2E testing when appropriate, e.g.
precommit. You can use something like Husky to easily manage the hooks.
You can also integrate the tests into your existing CI to run when a new release is created.
At this point, we have _some_ peace of mind that our plugin code doesn’t break the entire site. However, there’s a lot more we can do. We can start adding tests that validate our specific plugin’s functionality. Tests to check that logging in works fine, check that our plugin adds a menu item to the admin, check that our plugin’s settings page loads and saves settings properly, etc. Don’t forget “unhappy path” tests!
You can even validate functionality on specific browsers and specific viewport sizes. So, you can add tests that validate your mobile menu functionality separate from the desktop navigation.
In a future post, we’ll work with reusable fixtures, which are invaluable helpers when performing the same task or requiring a similar state across multiple tests.