There are many visual testing tools out there these days. Previously I have covered Applitools which without a doubt is great product with awesome people who contribute a lot to the testing community. I have also looked at Galen which is a more of a DOM layout tester. In this article we will have a look at BackstopJs, which is the closest opensource project I’ve come across that matches what Applitools do with regard to the way visual regressions are presented to the user.
So, in a nutshell:
BackstopJS automates visual regression testing of your responsive web UI by comparing DOM screenshots over time.
The good thing about Applitools is I can just dump in a line of code to already existing tests and see it do it’s thing. We have to start from the ground up here but fortunately the project maintainers have done a fantastic job of making everything streamline, so let’s go.
I’m a windows user in my day to day so lets have a look at setting this up. Note, you will require some pre-reqs in order to get started:
BackstopJs has very good documentation, so I’ll try not to just repeat what that covers. Rather, give a quick flavour of what BackstopJs can do and how to overcome any issues I had during set up.
There was only one of those actually, as I am looking at Local installation I edited my PATH environment variable to include backstop.cmd, so just had to add the following parameter: C:\Users\username\AppData\Roaming\npm\
It’s important to remember the workflow here, we need to call Backstop Init before we run our baseline tests.
- Backstop Init (from your project directory)
- Backstop test (get our base images)
- Backstop approve (set the base images)
- Backstop test – run the tests and compare against the baseline
To hit the ground running I recommend cloning this sample project and just tweaking the settings to meet your needs.
If we edit/ tweak ‘backstop.json’ we can easily dump new scenarios into the Json array once we are familiar with it, add or modify viewport sizes and exclude elements on the page that may be much more variable than others.
In the output we get a cool slider in the visual diff inspector, a scrubber for checking out differences and other useful info in a nicely styled report, tests are also very fast to execute.
This was great for quickly spinning up tests against 25 odd branded log in screens generating 75 test cases covering three different resolutions. Yes, it’s not the same as emulating or against physical devices but nonetheless useful.
Digging a little deeper
What if we do want to log in though? The recommended ‘engine’ is some selenium-esque coding which is required in the OnReady.js file for the specified engine. You can use others but here we’re looking at the recommended option of Puppeteer.
If you’re familiar with Selenium then Puppeteer is a breeze and also has very good API documentation. This is for use with Chromium only (makes sense as it’s created by the Google dev tools team).
To perform a login for example, we add some Puppeteer code:
console.log('SCENARIO > ' + scenario.label);
await require('./clickAndHoverHelper')(page, scenario);
// add more ready handlers here...
const navigationPromise = page.waitForNavigation();
await navigationPromise; // The navigationPromise resolves after navigation has finished
.waitForSelector('#homeSupport > h3:nth-child(8) > a')
.then(() => console.log('First URL with image: ' + currentURL));
for (currentURL of ['https://www.someUrl.com'])
await page.waitFor('#homeSupport > h3:nth-child(8) > a');
If you require an area to be excluded from the test such as some dynamic panel with variable data in it, we can hide DOM elements in the config (backstop.json).
Why not give BackstopJs a try? It could save you a lot of time.