Automated Tester

.science

This content shows Simple View

Selenium

Proxying UI Automation to OWASP ZAP

Quick disclaimer: I’m not a security expert, pen tester or ZAP expert but that doesn’t mean to say we should ignore security. A cheap way of adding a layer of security testing is to take your existing Selenium automation and proxy them through OWASP Zed Attack Proxy. So lets get started.

This is not a guide on how to use OWASP Zap and will not go into great configuration detail.

  • Download, install and start OWASP ZAP (Requires Java) either locally or on a VM
  • Install FoxyProxy which is a popular browser plugin. You can get it for Chrome or Firefox.
  • Install root level certificate

In ZAP go to Options > Local Proxies and set the Address and Port as desired. In this article I am running ZAP on a VM so I put the address of the VM in and set the port to 8999. Add an additional proxy of localhost:8999.

We can test this with FoxyProxy, setup a proxy to point to the VM on port 8999 and you should see the ZAP API front end interface.

Zap API Frontend

Next go to the API options in ZAP and add the VMs IP address to the list of addresses permitted to use the API. For the purposes of this article, I have also disabled the API key required to perform commands.

Now we can hit that API front end without the use of FoxyProxy.

So lets setup a connection to our ZAP instance in code and point tests via the proxy. Lets start by adding the OWASPZAPDotNetAPI nuget package.

We’ll connect to the ZAP service at the start of our test run in the hook:

public static ClientApi Zap;

[BeforeTestRun]
public static void BeforeTestRun()
{
// Note if you are using an API key, pass it in here instead of null
Zap = new ClientApi("192.168.xxx.xxx", 8999, null);

...
}

Now lets configure a proxy and pass it to the driver:

var options = new ChromeOptions(); var options = new ChromeOptions();

options.AddArgument("start-maximized");
options.AddArguments("disable-infobars");

// ZAP Proxy (Passive Scan) - ZAP should already be invoked.

var proxy = new Proxy();
options.Proxy = proxy;
proxy.Kind = ProxyKind.Manual;
proxy.IsAutoDetect = false;
proxy.HttpProxy = "192.168.xxx.xxx:8999";
proxy.SslProxy = "192.168.xxx.xxx:8999";

options.AddArgument("ignore-certificate-errors");
options.Proxy = proxy;
var timespan = TimeSpan.FromMinutes(3);
_driver = new RemoteWebDriver(new Uri(gridHub), options.ToCapabilities());

If we run a test, we should see traffic being generated in the ZAP History tab:

Zap Traffic

Great, we’re almost there. Now it’s time to generate a report – you can do this manually by using the API UI in core/other/html report (http://192.168.xxx.xxx:8999/UI/core/other/htmlreport/) but as we have a connection in code to the API lets do it there, after the test run:

public static void WriteZapHtmlReport(string path, byte[] bytes)
{
File.WriteAllBytes(path, bytes);
}

[AfterTestRun]
public static void AfterTestRun()
{
HookHelper.WriteZapHtmlReport(ReportDirectory + "_PassiveScanReport.html", Zap.core.htmlreport());
Zap.Dispose();

Here we take the byte array from the zap.core api and write it to a HTML file somewhere on disk, giving you a handy passive scan report.

Zap Report

Limitations

Be aware that any test that uses ZAP needs to have exclusive use of it- so to generalise, it’s better to keep all the browser based tests sequential. This includes tests that use ZAP as a proxy, not just the scanning tests. These words were robbed from here and are very true having spent a couple of hours see if the above would work with concurrency.



Surgical Strike UI Automation Testing

A lot of people when starting out with automated testing or Selenium may follow a kind of record and playback approach to writing automated tests, whether this is born out of using something like the plugin or just the general approach:

  • Fire up a browser
  • Goto the site
  • Login
  • Get to where you need to be
  • Perform a bunch of interactions
  • Logout (possibly)

There are a few optimisations we can do without much effort, like navigating with a url rather than clicking a bunch of menu items. This approach may need an environment with test data already present and that can turn into a big overhead. It might be a lot harder to ‘get to where you need to be’ if you have to create a whole structure first and doing that in the UI as part of your test should be avoided.

Trimming the fat

In the past on this blog I have talked about API tests and UI tests, lets combine the two to really optimise the UI tests. You’re writing a Selenium test to test a specific piece of functionality, lets keep it that way and just use Selenium to perform precision, surgical UI interactions that we care about. This will speed up your tests and make them more stable.

This way we will:

  • Perform a bunch of internal API calls to set the test up
  • Fire up a browser
  • Login
  • Navigate
  • Do the test
  • Perform another set of internal API calls to rip out the test setup

I’ve found a nice way to do this is to setup a stack which we can push fixtures onto and then iterativeley pop them off after the test is done.

private Stack<Action> teardownActions;

public Stack<Action> TeardownActions => teardownActions ?? (teardownActions = new Stack<Action>());

As usual I am using Specflow in my setup, now I turn to the hooks to perform an action I need for each test – lets say…. make a folder.

FolderRootDto = new FolderDto

{

FolderVisibility = 50,

Name = $"{ScenarioInfo.Title}"

};

Folders.CreateNewFolder(FolderRootDto.Name, FolderRootDto);

Stack it out

When the create method is called and we actually perform the api post request, we get the Id from the DTO (or whatever we need in the delete call) and push an action onto the stack, in this case another method that calls delete.

var client = client;

var newFolder = RestManager.Post<FolderDto>(client, BaseUrl + "/rest/endpoint/to/call", folderDto);

if (newFolder.StatusCode == HttpStatusCode.OK)

{

var LastFolderCreatedId = newFolder.Response.Id;

TreeState.Get(ScenarioContext).TeardownActions.Push(() =>

{

DeleteFolder(newFolder.Response.Id);

});

}

Now before our browser even fires, I have a folder to perform a test in, if we add our teardown action stack to fire after the scenario then this newly created folder gets ripped out afterwards.

var nextAction = TeardownActions.Pop();

while (nextAction != null)

{

nextAction();

nextAction = null;
if (TeardownActions.Count > 0)

{

nextAction = TreeState.Get(scenarioContext).TeardownActions.Pop();

}

}

As these actions get performed in milliseconds it can drastically reduce the time of your Selenium test that might be doing the all the foundation work or reduce the overhead you might have in order to get your fixture setup in place whether it be build steps or database restores.



Appium Mobile Emulation

Getting started with Appium requires a few installs and some configuration, so let’s get to it.

This article assumes you already have Java installed and set to the PATH environment variables.

What you need:

Android Studio

Once installed you need to look under Tools > Android > SDK. There are several components we need. Namely SDK Tools, SDK Build Tools and SDK Platform Tools.

Android SDK Install

Once setup you will need to add ANDROID_HOME to your PATH environment variable and point it to your main SDK folder. Note: A reboot may be required for Windows to pick up the change.

Android HOME

Visual Studio Emulator for Android

Now let’s look at Visual Studio Emulator for Android, once installed it should hopefully list a bunch of android API device profiles you can either run (if HyperV is enabled). Start one up and you should be able to interact with the virtual device. Note: If you look in Hyper-V Manager you can see the running emulator and tweak memory usage etc.

Device Profiles

HyperV Manager

If we navigate to our sdk tools we can also see the device running by running 

adb devices -L

 and this will list the device name for our Appium setup. In this instance the device is donatello.

Tip: If you happen to want to install an App you can do so with the command: adb install <path\to\yourapk.apk>

Appium Desktop

Next download Appium and click the Android icon to configure the server, input the relevant fields and device name then click the start button to run the server.

Appium Config

Installing an SSL Certificate on the device if required

As we are interacting with a website rather than an App for the purposes of this article, it can be useful to store SSL Certs on the devices when connecting. Using Visual Studio Emulator for Android we can setup and SD Card.

Emulator Options > SD Card

Pull from SD card first to a temp folder in order to create the directory structure first, then place the SSL Certificate in an easy to remember place and push to the SD card. Under Settings > Privacy in the Android OS we can then add a trusted certificate to the device.

When doing this, Android will force you to create a passcode for the device, do so and keep it easy to remember.

How do I find Android UI Elements for my tests?

Back over in Android SDK land…. in Sdk/tools/bin there is a batch file called uiautomatorviewer.bat. Open this in a text editor and find the line:

call "%java_exe%" "-Djava.ext.dirs=%javaextdirs%" "-Dcom.android.uiautomator.bindir=%prog_dir%" -jar %jarpath% %*

Then replace it with:

call "%java_exe%" "-Djava.ext.dirs=%javaextdirs%" "-Dcom.android.uiautomator.bindir=C:\DEV\androidSDK\tools" -jar %jarpath% %*

Making sure to set the binding directory to where your SDK tools are installed.

Then run uiautomatorviewer.bat whilst your emulator is running. Clicking the Android screenshot icon (second one from the left) will paint a view of what is currently on the emulator screen, you can then hover over buttons etc in order to find locators and anything else you might need.

uiautomatorviewer

Please note, your emulator API must be level 17 and upwards.

Time to Code

We can now write automation to run via this emulator with the usual webdriver goodies. Install the Appium Webdriver NuGet package and create the driver with the desired capabilities.

DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.SetCapability("device", "Android");
capabilities.SetCapability(CapabilityType.Platform, "Windows");
capabilities.SetCapability("deviceName", "donatello");
capabilities.SetCapability("platformName", "Android");
capabilities.SetCapability("platformVersion", "6.0.0");
capabilities.SetCapability(MobileCapabilityType.BrowserName, "BROWSER");
capabilities.SetCapability(CapabilityType.AcceptSslCertificates, true);

driver = new AndroidDriver<AndroidElement>(new Uri("http://127.0.0.1:4723/wd/hub"), capabilities, TimeSpan.FromSeconds(180));

Note that browserName “BROWSER” will start the devices’ native Android browser.

We can now start the automation of the site, starting with some very crude Selenium here… :

driver.Navigate().GoToUrl("https://mySecureUrl.com");

Thread.Sleep(1000);
var element = driver.FindElement(By.Id("Email"));

element.Click();
element.Clear();
element.SendKeys("username@mySecureUrl.com");

var element2 = driver.FindElement(By.Id("Password"));
element2.Click();
element2.Clear();
element2.SendKeys("mySecurePassword");
element2.SendKeys(Keys.Enter);

Hopefully this is a an easy enough to follow guide to get you up and running using Appium in your c# dev environment, these are the steps I took to being playing with Appium in more depth. Happy testing!



Automated Responsive Layout Testing with Galen Framework and Java

A short time ago Joe Colantonio wrote a good piece about the differences between Applitools and Galen. I had not heard of Galen Framework before so started to look at the CLI version. Seeing some value for a project I began to ponder how I could use it and get some high level feedback on a responsive site and get it fast.

So off I went looking for the C# bindings only to see there are none, so I decided to dip my toes in with Java as it is similar to C# (but you can also use JavaScript).

To my surprise I had something I could demo to my team in a couple of hours even though I’ve never touched Java or the IDE I downloaded (IntelliJ Community Edition). So what happened next?

I cloned the example java tests at the Galen repo located here. In order to build the project I had to install and reference a JRE from Java and that was that.

It was then just a case of hacking the example tests to suit what I was doing. Galen uses Selenium to spin up a driver of your choice and set the viewport size accordingly. If you are familiar with Selenium (I image most people coming here are) then it’s easy enough to manipulate it to do your logins and such, it is quite similar to C#.

The DSL

Galen has it’s own DSL, it’s not quite Gherkin but it is easy to get. You essentially declare the objects on the page you are interested in and write tests that say element A is 5px below element B and that this is true on all devices you have specified. All of which are declared in a *.spec or *.gspec file like so:

@objects
    banner-block        css     #HLHelp > header > div > h1
    search-box          id      searchBox
    side-block          css     #HLHelp > header > aside
    main-section        id      main
    header              css     #HLHelp > header
    getting-started     css     #main > article > section:nth-child(1)
    network             css     #main > article > section:nth-child(2)

= Header =
    @on mobile,tablet,laptop
        search-box:
            below banner-block width 15 to 30 px

    @on desktop
        search-box:
            below banner-block width 30 to 35 px

= Main =
    @on mobile,tablet,laptop
        getting-started:
            below header width 20px

    @on desktop
        getting-started:
            below header width 30px

    @on tablet, laptop
        getting-started:
            left-of network  width 5 to 10 px

The device names you can see are mapped from the GalenTestBase file, more on that soon. And if you get stuck with the options available or the syntax, you will be glad to know there is some very good documentation.

Devices / Viewports

Devices and their viewport sizes are easy to declare too:

<pre>@DataProvider(name = "devices")
public Object [][] devices () {
    return new Object[][] {
            {new TestDevice("mobile", new Dimension(360, 640), asList("mobile"))},
            {new TestDevice("tablet", new Dimension(768, 1024), asList("tablet"))},
            {new TestDevice("laptop", new Dimension(1366, 720), asList("laptop"))},
            {new TestDevice("desktop", new Dimension(1920, 1080), asList("desktop"))}
    };
}

Because Galen is Selenium based then of course you can run against your existing Grid, SauceLabs or BrowserStack configurations.

The Tests

Define a test name, load a URL, do some Selenium interactions (if required) and then check the layout from your spec file:

@Test(dataProvider = "devices")
public void popDown_shouldLookGood_onDevice(TestDevice device) throws IOException {
    load("/");
    getDriver().findElement(By.id("searchBox")).sendKeys("WiFi");
    checkLayout("/specs/loginPage.spec", device.getTags());
 }

The Output

After it has run Galen produces a HTML report which is very handy and also produces heat maps for the elements declared in your gspec file. The Galen website has a good example HTML report here.

So what’s left? Hook it up to your CI and away you go. Although you if plan on using this more extensively than something high level you may want to structure your solution accordingly and implement a more page object approach.

Summary

Galen has its uses and may be a viable high level alternative if you cannot get hold of an Applitools license, but as Joe states in his article there are many differences between the two. Although Galen has a built in image comparison tool, I have no idea how to set this up yet. Either way I got something quick (if dirty) working in no time and furthermore I found myself quite pleased/ surprised with how easy it was to use.



Concurrent Test Running with Specflow and NUnit 3

A little while back I wrote a post on achieving concurrent or parallel test running with Selenium, Specflow and NUnit2, but what about NUnit3? Let’s have a look at that, thankfully it is a bit simpler than running an external build script as done previously.

First up, according to the Specflow docs – open up your AssemblyInfo.cs file in your project and add the following line:

[assembly: Parallelizable(ParallelScope.Fixtures)]

Next we need to setup/ teardown our browser and config from the Scenario Hook level, doing it in a Test Run Hook will be problematic as it is static. Your Hooks might look similar to this:

[Binding]
public class ScenarioHooks
{
private readonly IObjectContainer objectContainer;
private IWebDriver Driver;

public ScenarioHooks(IObjectContainer objectContainer)
{
this.objectContainer = objectContainer;
}

[BeforeScenario]
public void BeforeScenario()
{
var customCapabilitiesIE = new DesiredCapabilities();
customCapabilitiesIE.SetCapability(CapabilityType.BrowserName, "internet explorer");
customCapabilitiesIE.SetCapability(CapabilityType.Platform, new Platform(PlatformType.Windows));
customCapabilitiesIE.SetCapability("webdriver.ie.driver", @"C:\tmp\webdriver\iedriver\iedriver_3.0.0_Win32bit.exe");

Driver = new RemoteWebDriver(new Uri(XXX.XXX.XXX.XXX), customCapabilitiesIE);
objectContainer.RegisterInstanceAs<IWebDriver>(Driver);
}

[AfterScenario]
public void AfterScenario()
{
Driver.Dispose();
Driver.Quit();
}
}

You can see from the browser instantiation we are sending the tests to a Selenium Grid Hub, so as a precursor to running the tests you will need suitable infrastructure to run a grid, or you could configure it to go off to SauceLabs or BrowserStack.

Assuming the hub and nodes are configured correctly, when your build process runs the tests then the hub will farm them out by feature file (for other options see the parallel scope in AssemblyInfo.cs) to achieve concurrent test running, and that’s it! Much nicer.




top