Top 5 Visual Regression Testing Tools for WordPress
The importance of a good website UX cannot be stressed enough. Regardless of the size and scope of the website – whether it’s a simple blog, a creative portfolio or a business website – your visitors must enjoy a smooth and streamlined experience. To assure that, all design elements and functionalities on your website need to work properly and to be in sync with each other. There are many ways to check for this, and today we want to talk about visual regression testing tools for WordPress.
First, let’s see what visual regression testing (VTR) tools are and why you may want to use one.
Qode Themes: Top Picks
View Collection
What Is Visual Regression Testing and Why Do You Need It?
In short, visual regression testing is a method that reveals changes in your design, or in your frontend in general, that you may want to fix. It identifies changes, various bugs and potential issues that would otherwise remain undiscovered by general testing tools because they’re either too small or too subtle, or are otherwise capable of escaping these tests.
In most cases, visual regression testing works by capturing a sequence of screenshots of your frontend during a thorough scan. After the initial sequence, the tool then performs additional scans and also takes images which are then compared to the initial ones in order to detect changes or bugs.
Some of the issues you can catch this way include misalignments of page sections and other design elements (e.g. a menu that, when opened, covers the title area so that parts of it are hidden), mistakes in responsive design (i.e. elements that don’t work well on smaller screens), improper content loading, failed API integrations, as well as any larger issues that occur due to updates.
Speaking of updates, they are the main reason you may want to conduct visual regression testing. After updating your theme, your plugins or core files, there’s always a risk of losing some important element of your site or, simply put, of breaking it. A WordPress update can also cause certain issues, especially if you haven’t properly backed up your website. One way to detect this is by manually going through all your pages, but that’s simply not convenient enough. By using VTR tools, this process is automated and also much more reliable.
Now that we understand the importance of VTR, let’s check out some of the best visual regression testing tools for your WordPress website:
Wraith is a visual testing tool based on screenshot comparisons, developed by none less than the BBC News team. This open-source and free tool which can be found on GitHub uses a headless browser to capture the pages at different times or in different environments and then compares them. Any difference that is found between the two is highlighted in blue.
This tool can run in two modes: comparison of two domains (which can involve spidering of two domains for changes or running several comparisons at once) and comparison of the same domain over different time intervals.
After the tool takes the screenshots, it runs a comparison and creates both an image with differences highlighted, and a .txt with percentages of pixels that differ. The user can specify a threshold for changes and differences and if the test shows results that are above the threshold, it will yield a system error code.
Wraith is free but it does require installation and scripting. You will also have to use at least one headless browser, such as Chrome, CasperJS, SlimerJS or PhantomJS. You will also need to have ImageMagick installed in order to view the comparison files.
BackstopJS is a visual regression testing tool for responsive web apps and another open-source tool on our list. Similarly to Wraith, it requires you to have a solid grasp of installation and scripting, and you will need to have Chrome , CasperJS or another headless browser.
BackstopJS provides users with in-browser reporting UI with separate layout settings for print and screen, scenario display filtering, test approval, reference, test and visual diff inspector, and something that the developers call “cool scrubber thingy.” You also get CLI reports, integrated docker rendering that prevents cross-platform rendering, JUnit reports, and render tests with Chrome Headless.
On top of that, you get to simulate user interactions using Puppeteer or Playwright scripts. Plus, the tool has only three commands and, except for installation and scripting, it’s very easy to use.
If you’re looking for a VRT tool that’s built with collaboration in mind, you should check out Percy. This excellent tool was developed by BrowserStack and lists Twitter, Microsoft, jQuery, Harvard University and Wikimedia as its most prominent users.
This tool is designed to help you detect visual bugs by capturing screenshots and then conducting pixel-to-pixel comparisons with every commit. So far, it’s basically the same thing as the previous tools, except Percy has some additional automation and integration features that allow you to seamlessly incorporate it into your workflow. You can get results either in your test integration framework or directly in your application. Visual changes are grouped and filtered to exclude noisy elements, which guarantees faster reviews. Your team is informed of everything every step of the way thanks to notifications and comments functionality.
You can use Percy core features for free, with unlimited users and up to 5000 screenshots per month. If you need more (and richer features), there are paid plans for desktop, desktop and mobile, and enterprise version, for which you’ll need to contact the BrowserStack sales team.
WordPress users looking for a visual regression testing tool designed specifically for their CMS should check out VRTs – VIsual Regression Test. It is a plugin that, once installed and activated, regularly checks your website’s frontend for visual bugs and changes and then sends alerts and notifications in your backend. It works by using a reference screenshot, set by the user, and daily automated screenshots. It then compares them with each other to detect changes. It can help with bugs and issues caused by plugin updates, missing checks, code changes, server problems, malicious code, external services such as incorrectly fetched API integrations, and so on.
This solution is ideal for WordPress users who don’t want to bother with installation and scripting, since it requires no coding and is very easy to set up.
The free version of this plugin offers testing of three pages max, daily testing intervals and alert history up to a month. The Pro plan costs $39/month and allows you to test up to 25 pages, has a 90 day alert history, multiple alert participants and email support.
Another code-free solution for UI VTR testing you should check out is Screenster. It is a somewhat different solution from the others on this list since it doesn’t take screenshots and instead records your interactions with your webpages. It then reruns the recorded tests and optimizes them with automatically extracted parameters and also stores self-healing selectors for all actions. After the test, you can review the detected changes and update your default baseline with them. If, for whatever reason, you need to exclude one or more UI elements from the comparison, that’s possible, too. There’s also a Team Portal that your team can use to run tests and access the results.
This is a server-based solution, which means you don’t have to install anything or adjust anything at all. Simply paste the desired URL to Screenster and let it run a 5-minute test. You can do this on the Screenster cloud or download the server and install it on-prem. While this is definitely an advantage that Screenster has over the other VTR solutions, it also has a downside – it has a steep learning curve, as it requires you to be comfortable with working with servers.
Screenster has a free plan with a single concurrent test, one user max, limited cloud test duration (3 minutes), and community support. Paid plans start at $25 per month and include multiple concurrent tests, more users, more tests on local and on cloud, longer test duration, and JIRA integration.
Wrapping It Up
We’re pretty confident that the solution you need for visual regression testing of your UI is right here on this list. You want a free and open-source solution that captures and compares screenshots? PIck between Wraith and BackstopJS. You prefer a WordPress plugin? For you, there’s VRTs – Visual Regressions Test. Percy is great for teamwork and automation, and Screenster is perfect for bigger websites with enterprise needs, but also for smaller teams that are comfortable with server environments and want a feature-rich solution.
Let us know which one you picked!
