Automated Tester

.science

This content shows Simple View

Visual testing with BackstopJs

There are many visual testing tools out there these days. Previously I have covered Applitools which without a doubt is great product with awesome people who contribute a lot to the testing community. I have also looked at Galen which is a more of a DOM layout tester. In this article we will have a look at BackstopJs, which is the closest opensource project I’ve come across that matches what Applitools do with regard to the way visual regressions are presented to the user.

So, in a nutshell:

BackstopJS automates visual regression testing of your responsive web UI by comparing DOM screenshots over time.

Setup

The good thing about Applitools is I can just dump in a line of code to already existing tests and see it do it’s thing. We have to start from the ground up here but fortunately the project maintainers have done a fantastic job of making everything streamline, so let’s go.

I’m a windows user in my day to day so lets have a look at setting this up. Note, you will require some pre-reqs in order to get started:

BackstopJs has very good documentation, so I’ll try not to just repeat what that covers. Rather, give a quick flavour of what BackstopJs can do and how to overcome any issues I had during set up.

There was only one of those actually, as I am looking at Local installation I edited my PATH environment variable to include backstop.cmd, so just had to add the following parameter: C:\Users\username\AppData\Roaming\npm\

Workflow

It’s important to remember the workflow here, we need to call Backstop Init before we run our baseline tests.

• Backstop Init (from your project directory)
• Backstop test (get our base images)
• Backstop approve (set the base images)
• Backstop test – run the tests and compare against the baseline

To hit the ground running I recommend cloning this sample project and just tweaking the settings to meet your needs.

If we edit/ tweak ‘backstop.json’ we can easily dump new scenarios into the Json array once we are familiar with it, add or modify viewport sizes and exclude elements on the page that may be much more variable than others.

Output

In the output we get a cool slider in the visual diff inspector, a scrubber for checking out differences and other useful info in a nicely styled report, tests are also very fast to execute.

This was great for quickly spinning up tests against 25 odd branded log in screens generating 75 test cases covering three different resolutions. Yes, it’s not the same as emulating or against physical devices but nonetheless useful.

Digging a little deeper

What if we do want to log in though? The recommended ‘engine’ is some selenium-esque coding which is required in the OnReady.js file for the specified engine. You can use others but here we’re looking at the recommended option of Puppeteer.

If you’re familiar with Selenium then Puppeteer is a breeze and also has very good API documentation. This is for use with Chromium only (makes sense as it’s created by the Google dev tools team).

module.exports = async (page, scenario, vp) => {
console.log('SCENARIO > ' + scenario.label);
await require('./clickAndHoverHelper')(page, scenario);

await page.focus('#Email')
await page.keyboard.type('email@someDomain.com');
await page.keyboard.press('Enter');

page
.waitForSelector('#homeSupport > h3:nth-child(8) > a')
.then(() => console.log('First URL with image: ' + currentURL));
for (currentURL of ['https://www.someUrl.com'])
await page.goto(currentURL);
await page.waitFor('#homeSupport > h3:nth-child(8) > a');
};

If you require an area to be excluded from the test such as some dynamic panel with variable data in it, we can hide DOM elements in the config (backstop.json).

Why not give BackstopJs a try? It could save you a lot of time.

Appium Mobile Emulation

Getting started with Appium requires a few installs and some configuration, so let’s get to it.

What you need:

Android Studio

Once installed you need to look under Tools > Android > SDK. There are several components we need. Namely SDK Tools, SDK Build Tools and SDK Platform Tools.

Once setup you will need to add ANDROID_HOME to your PATH environment variable and point it to your main SDK folder. Note: A reboot may be required for Windows to pick up the change.

Visual Studio Emulator for Android

Now let’s look at Visual Studio Emulator for Android, once installed it should hopefully list a bunch of android API device profiles you can either run (if HyperV is enabled). Start one up and you should be able to interact with the virtual device. Note: If you look in Hyper-V Manager you can see the running emulator and tweak memory usage etc.

If we navigate to our sdk tools we can also see the device running by running

and this will list the device name for our Appium setup. In this instance the device is donatello.

Tip: If you happen to want to install an App you can do so with the command: adb install <path\to\yourapk.apk>

Appium Desktop

Next download Appium and click the Android icon to configure the server, input the relevant fields and device name then click the start button to run the server.

Installing an SSL Certificate on the device if required

As we are interacting with a website rather than an App for the purposes of this article, it can be useful to store SSL Certs on the devices when connecting. Using Visual Studio Emulator for Android we can setup and SD Card.

Pull from SD card first to a temp folder in order to create the directory structure first, then place the SSL Certificate in an easy to remember place and push to the SD card. Under Settings > Privacy in the Android OS we can then add a trusted certificate to the device.

When doing this, Android will force you to create a passcode for the device, do so and keep it easy to remember.

How do I find Android UI Elements for my tests?

Back over in Android SDK land…. in Sdk/tools/bin there is a batch file called uiautomatorviewer.bat. Open this in a text editor and find the line:

call "%java_exe%" "-Djava.ext.dirs=%javaextdirs%" "-Dcom.android.uiautomator.bindir=%prog_dir%" -jar %jarpath% %*

Then replace it with:

call "%java_exe%" "-Djava.ext.dirs=%javaextdirs%" "-Dcom.android.uiautomator.bindir=C:\DEV\androidSDK\tools" -jar %jarpath% %*

Making sure to set the binding directory to where your SDK tools are installed.

Then run uiautomatorviewer.bat whilst your emulator is running. Clicking the Android screenshot icon (second one from the left) will paint a view of what is currently on the emulator screen, you can then hover over buttons etc in order to find locators and anything else you might need.

Time to Code

We can now write automation to run via this emulator with the usual webdriver goodies. Install the Appium Webdriver NuGet package and create the driver with the desired capabilities.

DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.SetCapability("device", "Android");
capabilities.SetCapability(CapabilityType.Platform, "Windows");
capabilities.SetCapability("deviceName", "donatello");
capabilities.SetCapability("platformName", "Android");
capabilities.SetCapability("platformVersion", "6.0.0");
capabilities.SetCapability(MobileCapabilityType.BrowserName, "BROWSER");
capabilities.SetCapability(CapabilityType.AcceptSslCertificates, true);

driver = new AndroidDriver<AndroidElement>(new Uri("http://127.0.0.1:4723/wd/hub"), capabilities, TimeSpan.FromSeconds(180));

Note that browserName “BROWSER” will start the devices’ native Android browser.

We can now start the automation of the site, starting with some very crude Selenium here… :

driver.Navigate().GoToUrl("https://mySecureUrl.com");

var element = driver.FindElement(By.Id("Email"));

element.Click();
element.Clear();

element2.Click();
element2.Clear();
element2.SendKeys(Keys.Enter);

Hopefully this is a an easy enough to follow guide to get you up and running using Appium in your c# dev environment, these are the steps I took to being playing with Appium in more depth. Happy testing!

Use Selenium to Automate Windows with WinAppDriver

If you have ever found yourself in the position that you needed to automate a Windows application as part of your day to day automation testing you might well have come across AutoIt or gone down the CodedUI path. Now however, those good folk at Microsoft have made WinAppDriver which makes use of all the usual Selenium goodies… from the horses mouth:

Windows Application Driver is a service to support UI Test Automation of Windows Applications. The service design subscribes to the Mobile JSON Wire Protocol standard. If you’ve been looking for better support for using Appium to test Windows Applications then this service is for you!

The GitHub page has some good code samples to get you started (so I won’t bore you with any of mine), such as manipulating the Windows calculator. As you will see though the driver is still in beta but has come a long way recently and to help my foray into using it it now supports WCF applications.

Finding locators is pretty easy with the tool they suggest (Inspect) which is not too dissimilar to Spy++ if you have ever used that. Further to that I have found that the accessibility id locator works best for me (so far anyway!).

After installing the WinAppDriver executable, you can also run remotely which looks good but it does seem to only like being run on Windows 10 so you may need to take that into account if you don’t have the resources already in place.

If you decide to give it a whirl and encounter problems I would have a look at the issues page – chances are someone else has encountered it and a fix may already be underway.

Now obviously with WinAppDriver, this is only ever going to work with windows so I wouldn’t ever go recommending that this is used in the same project as your usual Selenium web based tests.

Until next time, have fun experimenting with WinAppDriver.

Mobile Emulation with ChromeDriver

We can use ChromeDriver to run automated Selenium tests against a whole stack of different devices with Chrome Dev Tools Mobile Emulation option.

Devices currently supported at the time of writing:

• Amazon Kindle Fire HDX
• Apple iPhone 4, 5, 6 & 6+
• Blackberry Playbook
• Blackberry Z30
• Google Nexus 4, 5, 6, 7 & 10
• LG Optimus L70
• Laptop with HiDPL Screen
• Laptop with MDPI Screen
• Laptop with touch
• Nokia Lumia 520
• Nokia N9
• Samsung Galaxy Note 3
• Samsung Galaxy Note II
• Samsung Galaxy Note S III
• Samsung Galaxy S4
Discovering the options available

Now it’s just a case of passing the device string to ChromeOptions’ EnableMobileEmulation option when instantiating the driver, like so: