Automated Tester

.science

This content shows Simple View

Applitools

Automated Responsive Layout Testing with Galen Framework and Java

A short time ago Joe Colantonio wrote a good piece about the differences between Applitools and Galen. I had not heard of Galen Framework before so started to look at the CLI version. Seeing some value for a project I began to ponder how I could use it and get some high level feedback on a responsive site and get it fast.

So off I went looking for the C# bindings only to see there are none, so I decided to dip my toes in with Java as it is similar to C# (but you can also use JavaScript).

To my surprise I had something I could demo to my team in a couple of hours even though I’ve never touched Java or the IDE I downloaded (IntelliJ Community Edition). So what happened next?

I cloned the example java tests at the Galen repo located here. In order to build the project I had to install and reference a JRE from Java and that was that.

It was then just a case of hacking the example tests to suit what I was doing. Galen uses Selenium to spin up a driver of your choice and set the viewport size accordingly. If you are familiar with Selenium (I image most people coming here are) then it’s easy enough to manipulate it to do your logins and such, it is quite similar to C#.

The DSL

Galen has it’s own DSL, it’s not quite Gherkin but it is easy to get. You essentially declare the objects on the page you are interested in and write tests that say element A is 5px below element B and that this is true on all devices you have specified. All of which are declared in a *.spec or *.gspec file like so:

@objects
    banner-block        css     #HLHelp > header > div > h1
    search-box          id      searchBox
    side-block          css     #HLHelp > header > aside
    main-section        id      main
    header              css     #HLHelp > header
    getting-started     css     #main > article > section:nth-child(1)
    network             css     #main > article > section:nth-child(2)

= Header =
    @on mobile,tablet,laptop
        search-box:
            below banner-block width 15 to 30 px

    @on desktop
        search-box:
            below banner-block width 30 to 35 px

= Main =
    @on mobile,tablet,laptop
        getting-started:
            below header width 20px

    @on desktop
        getting-started:
            below header width 30px

    @on tablet, laptop
        getting-started:
            left-of network  width 5 to 10 px

The device names you can see are mapped from the GalenTestBase file, more on that soon. And if you get stuck with the options available or the syntax, you will be glad to know there is some very good documentation.

Devices / Viewports

Devices and their viewport sizes are easy to declare too:

<pre>@DataProvider(name = "devices")
public Object [][] devices () {
    return new Object[][] {
            {new TestDevice("mobile", new Dimension(360, 640), asList("mobile"))},
            {new TestDevice("tablet", new Dimension(768, 1024), asList("tablet"))},
            {new TestDevice("laptop", new Dimension(1366, 720), asList("laptop"))},
            {new TestDevice("desktop", new Dimension(1920, 1080), asList("desktop"))}
    };
}

Because Galen is Selenium based then of course you can run against your existing Grid, SauceLabs or BrowserStack configurations.

The Tests

Define a test name, load a URL, do some Selenium interactions (if required) and then check the layout from your spec file:

@Test(dataProvider = "devices")
public void popDown_shouldLookGood_onDevice(TestDevice device) throws IOException {
    load("/");
    getDriver().findElement(By.id("searchBox")).sendKeys("WiFi");
    checkLayout("/specs/loginPage.spec", device.getTags());
 }

The Output

After it has run Galen produces a HTML report which is very handy and also produces heat maps for the elements declared in your gspec file. The Galen website has a good example HTML report here.

So what’s left? Hook it up to your CI and away you go. Although you if plan on using this more extensively than something high level you may want to structure your solution accordingly and implement a more page object approach.

Summary

Galen has its uses and may be a viable high level alternative if you cannot get hold of an Applitools license, but as Joe states in his article there are many differences between the two. Although Galen has a built in image comparison tool, I have no idea how to set this up yet. Either way I got something quick (if dirty) working in no time and furthermore I found myself quite pleased/ surprised with how easy it was to use.



Applitools Eyes: Easy Automated Visual Checking

Applitools Eyes allows us to easily integrate automated visual testing into our existing BDD Solution. Having automated visual testing is a big plus because whilst our automation may be good at asserting if elements are visible on the page or if some certain text is present, it doesn’t tell us that it has appeared in the right place.

Automated visual testing will capture baseline images to make future checks against.

To quote another article: “For example, a single automated visual test will look at a page and assert that every element on it has rendered correctly. Effectively checking hundreds of things and telling you if any of them are out of place. This will occur every time the test is run, and it can be scaled to each browser, operating system, and screen resolution you require.”

If you run this in parallel then you have a powerful testing beast on your hands.

Put another way, one automated visual test is worth hundreds of assertions. And if done in the service of an iterative development workflow, then you’re one giant leap closer to Continuous Delivery. — Dave Haeffner

As with everything on this blog it is tailored around C# .Net but the following works in a similar way with other bindings like Java, Python, Ruby and JS.

How it’s set up in the Solution for C# .Net

First off we need a Nuget package for Eyes:

Nuget_Eyes

Then in our TestRunContext Class we need to instantiate it as part of our window setup:

using Applitools;

private static Eyes eyes;
public static void WindowSetup()

{
    Driver.Navigate().GoToUrl("about:blank");
    Driver.Manage().Window.Maximize();

// This is your api key, make sure you use it in all your tests.
    eyes = new Eyes {ApiKey = "API KEY GOES HERE"};

// Start visual testing with browser.
// Make sure to use the returned driver from this point on.
   Driver = eyes.Open(Driver, "Application name", "Test Name", new System.Drawing.Size(1024, 768));
}

We call on the following in our tests to perform a check of the page, passing in a string to be displayed in Applitools:

public static void EyeCheck(string pageName)
        {
            // Visual validation point
            eyes.CheckWindow(pageName);
        }

And then close the Eyes at the end of our checks:

public static void CloseEyes()
        {
 // End visual testing. Validate visual correctness.
            eyes.Close();
            eyes.AbortIfNotClosed();
        }

Rather than storing the Selenium instance in the Driver variable, we’re now storing it in a local browser variable and passing it into eyes.Open — storing the WebDriver object that eyes.Openreturns in the Driver variable instead.

This way the Eyes platform will be able to capture what our test is doing when we ask it to capture a screenshot. The Selenium actions in our test will not need to be modified.

Before calling eyes.Open we provide the API key. When calling eyes.Open, we pass it the Selenium instance, the name of the app we’re testing (e.g., “Test Suite“), and the name of the test (e.g., “Test Name“) in the above example the part that says “new System.Drawing.Size(1024, 768)” is specifying the view port size of the browser.

For mobile testing it is not relevant, since the device size will be used, but for web testing, it is highly recommended to configure a size in order to ensure that the result of the test will be consistent even when you run it on different machines with different screen resolutions/sizes.  You can also paramterise it and run on several sizes in case of responsive design, but in general it is recommended to set it explicitly.

Now we’re ready to add some visual checks to our test by calling the following in our step definitions.

Page.PerformEyeCheck(pageName);

With eyes.CheckWindow() working in the background we are specifying when in the test’s workflow we’d like Applitools Eyes to capture a screenshot (along with some description text).

NOTE: These visual checks are effectively doing the same work as the pre-existing assertion (e.g. where we’re asking Selenium if a success notification is displayed and asserting on the Boolean result) — in addition to reviewing other visual aspects of the page. So once we verify that our test is working correctly we can remove this assertion and still be covered.

We end the test with eyes.Close. You may feel the urge to place this in teardown, but in addition to closing the session with Eyes, it acts like an assertion. If Eyes finds a failure in the app (or if a baseline image approval is required), then eyes.Close will throw an exception; failing the test. So it’s best suited to live in the test itself.

NOTE: An exception from eyes.Close will include a URL to the Applitools Eyes job in your test output. The job will include screenshots from each test step and enable you to play back the keystrokes and mouse movements from your Selenium tests. You can also determine the level of strictness to use or exclude areas of your page from being checked.

When an exception gets thrown by eyes.Close, the Eyes session will close. But if an exception occurs before eyes.Close can fire, the session will remain open. To handle that, we’ll need to add an additional command to our teardown.

eyes.AbortIfNotClosed(); will make sure the Eyes session terminates properly regardless of what happens in the test.

Now when we run the test, it will execute locally while also performing visual checks in Applitools Eyes.

Applitools Eyes

Further Example:

https://eyes.applitools.com/app/tutorial.html




top