Automated Tester

.science

This content shows Simple View

Uncategorized

Checking link status in C#

Sometimes you need to test an application that has external links or have the need to check links and it doesn’t make a whole lot of sense to physically visit these places and verify that something has loaded on it because it is way out of your control and that site could change at any given moment, making your tests flaky.

One way to circumvent this is to check the link status code of a given link by sending a http request and returning the status code. We can then parse this to a string and make an assertion based against it. The code below does all the heavy lifting.

        public HttpStatusCode GetHeaders(string url)
        {
            HttpStatusCode result = default (HttpStatusCode );

            var request = HttpWebRequest.Create(url);
            request.Method = "HEAD";
            using ( var response = request.GetResponse() as HttpWebResponse)
            {
                if (response != null)
                {
                    result = response.StatusCode;
                    response.Close();
                }
            }

            return result;
        }

So in our step definition all we need to do is assert the returned status code against what we are expecting it to be whether it be a 200 OK, 404, 500 etc.

We can now call this and make an assertion on the retrieved link status in our step definition files.

        var expectedStatus = VerifyPages.HttpStatusOk;
        var facebookLink = UrlConsts.Facebook;
        var facebookStatus = Page.GetHeaders(facebookLink).ToString();
        Assert.AreEqual(expectedStatus, facebookStatus);


Browser Profiles with Selenium

I decided to write this article after someone asked about browser profiles with Selenium in the IRC channel.

With Selenium we can specify a particular browser profile when instantiating our driver so if you happen to have specific browser setup requirements such as Firefox with certain plugins installed we can tell the driver to use this.

Setup

To begin with we will want to setup up our profile to be used. In this example we will use Firefox and use the Firefox profile manager.

To access this run “firefox -p” from Run.

profiles1

Which will load up the profile manager, here we will create a new profile and call it ‘QAToolsProfile’ – for this example we will choose the save folder as ‘Browser Profile’ on the desktop.

profiles2

profiles3

Next, start up firefox and add in the tools required for your new profile, I am going to add a tool called REST Client.

profiles4

profiles5

Right then! We now have a customised browser profile to call in our code.

Calling the Profile

As you might expect this is relatively easy as we just need to pass in the location of the profile which we dumped on the desktop:

private static void SetupFirefoxDriver()
{
string profileDir;

profileDir = @"C:\Users\user.name\Desktop\BrowserProfile";

FirefoxProfile profile = new FirefoxProfile(profileDir);
Driver = new FirefoxDriver(profile);
}

If we now say to a test to navigate to “chrome://restclient/content/restclient.html” we can automate the browser plugin. In this instance we can perform some API checks with RESTClient’s user interface. If we did not specify the profile then Firefox would load up with no extra frills.

profiles6

You can find more information on Browser Profiles with Firefox here.



Applitools Eyes: Easy Automated Visual Checking

Applitools Eyes allows us to easily integrate automated visual testing into our existing BDD Solution. Having automated visual testing is a big plus because whilst our automation may be good at asserting if elements are visible on the page or if some certain text is present, it doesn’t tell us that it has appeared in the right place.

Automated visual testing will capture baseline images to make future checks against.

To quote another article: “For example, a single automated visual test will look at a page and assert that every element on it has rendered correctly. Effectively checking hundreds of things and telling you if any of them are out of place. This will occur every time the test is run, and it can be scaled to each browser, operating system, and screen resolution you require.”

If you run this in parallel then you have a powerful testing beast on your hands.

Put another way, one automated visual test is worth hundreds of assertions. And if done in the service of an iterative development workflow, then you’re one giant leap closer to Continuous Delivery. — Dave Haeffner

As with everything on this blog it is tailored around C# .Net but the following works in a similar way with other bindings like Java, Python, Ruby and JS.

How it’s set up in the Solution for C# .Net

First off we need a Nuget package for Eyes:

Nuget_Eyes

Then in our TestRunContext Class we need to instantiate it as part of our window setup:

using Applitools;

private static Eyes eyes;
public static void WindowSetup()

{
    Driver.Navigate().GoToUrl("about:blank");
    Driver.Manage().Window.Maximize();

// This is your api key, make sure you use it in all your tests.
    eyes = new Eyes {ApiKey = "API KEY GOES HERE"};

// Start visual testing with browser.
// Make sure to use the returned driver from this point on.
   Driver = eyes.Open(Driver, "Application name", "Test Name", new System.Drawing.Size(1024, 768));
}

We call on the following in our tests to perform a check of the page, passing in a string to be displayed in Applitools:

public static void EyeCheck(string pageName)
        {
            // Visual validation point
            eyes.CheckWindow(pageName);
        }

And then close the Eyes at the end of our checks:

public static void CloseEyes()
        {
 // End visual testing. Validate visual correctness.
            eyes.Close();
            eyes.AbortIfNotClosed();
        }

Rather than storing the Selenium instance in the Driver variable, we’re now storing it in a local browser variable and passing it into eyes.Open — storing the WebDriver object that eyes.Openreturns in the Driver variable instead.

This way the Eyes platform will be able to capture what our test is doing when we ask it to capture a screenshot. The Selenium actions in our test will not need to be modified.

Before calling eyes.Open we provide the API key. When calling eyes.Open, we pass it the Selenium instance, the name of the app we’re testing (e.g., “Test Suite“), and the name of the test (e.g., “Test Name“) in the above example the part that says “new System.Drawing.Size(1024, 768)” is specifying the view port size of the browser.

For mobile testing it is not relevant, since the device size will be used, but for web testing, it is highly recommended to configure a size in order to ensure that the result of the test will be consistent even when you run it on different machines with different screen resolutions/sizes.  You can also paramterise it and run on several sizes in case of responsive design, but in general it is recommended to set it explicitly.

Now we’re ready to add some visual checks to our test by calling the following in our step definitions.

Page.PerformEyeCheck(pageName);

With eyes.CheckWindow() working in the background we are specifying when in the test’s workflow we’d like Applitools Eyes to capture a screenshot (along with some description text).

NOTE: These visual checks are effectively doing the same work as the pre-existing assertion (e.g. where we’re asking Selenium if a success notification is displayed and asserting on the Boolean result) — in addition to reviewing other visual aspects of the page. So once we verify that our test is working correctly we can remove this assertion and still be covered.

We end the test with eyes.Close. You may feel the urge to place this in teardown, but in addition to closing the session with Eyes, it acts like an assertion. If Eyes finds a failure in the app (or if a baseline image approval is required), then eyes.Close will throw an exception; failing the test. So it’s best suited to live in the test itself.

NOTE: An exception from eyes.Close will include a URL to the Applitools Eyes job in your test output. The job will include screenshots from each test step and enable you to play back the keystrokes and mouse movements from your Selenium tests. You can also determine the level of strictness to use or exclude areas of your page from being checked.

When an exception gets thrown by eyes.Close, the Eyes session will close. But if an exception occurs before eyes.Close can fire, the session will remain open. To handle that, we’ll need to add an additional command to our teardown.

eyes.AbortIfNotClosed(); will make sure the Eyes session terminates properly regardless of what happens in the test.

Now when we run the test, it will execute locally while also performing visual checks in Applitools Eyes.

Applitools Eyes

Further Example:

https://eyes.applitools.com/app/tutorial.html



Parallel Test Execution with Specflow

Running Specflow scenarios in Parallel is a tricky problem to solve due to the nature of how tests are written. Achieving this is a lot simpler if we write Selenium tests in a unit style fashion but as this isn’t the case for Specflow with its Gherkin syntax we will look at executing them with a powershell build script against a Browserstack grid.

Thanks to this great article from Kenneth Truyers, it has really helped me to achieve this. We will look at executing parallel Specflow tests against a BrowserStack grid.

Set Up

In our Initialisation method we need to specify our remote driver whilst passing in our varying desired capabilities in Config.

private static void SetupCloudDriver()
        {
            var capabilities = new DesiredCapabilities();

            capabilities.SetCapability(CapabilityType.Version, ConfigurationManager.AppSettings["version"]);
            capabilities.SetCapability("os", ConfigurationManager.AppSettings["os"]);
            capabilities.SetCapability("os_version", ConfigurationManager.AppSettings["os_version"]);
            capabilities.SetCapability("browserName", ConfigurationManager.AppSettings["browser"]);

            capabilities.SetCapability("browserstack.user", ConfigurationManager.AppSettings["browserstack.user"]);
            capabilities.SetCapability("browserstack.key", ConfigurationManager.AppSettings["browserstack.key"]);
            capabilities.SetCapability("browserstack.local", false);
            capabilities.SetCapability("browserstack.debug", true);

            capabilities.SetCapability("project", "Project Name");

            Driver = new RemoteWebDriver(new Uri(ConfigurationManager.AppSettings["browserstack.hub"]), capabilities);
            Driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(1));
            ScenarioContext.Current["driver"] = Driver;
        }

Config for Cross Browser spin up

One of our config files might look like so:

<xml version="1.0" encoding="utf-8">
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
    <appSettings>
        <add key="browser" value="Safari" xdt:Transform="Insert"/>
        <add key="os" value="osx" xdt:Transform="Insert"/>
        <add key="version" value="8" xdt:Transform="Insert"/>
        <add key="os_version" value="Yosemite" xdt:Transform="Insert"/>
    </appSettings>
</configuration>

This tells Browserstack what system to spin up and run the tests against for one parallel instance, multiple config files with other systems are needed so they can all be executed depending on your requirements. Furthermore we need to set these up in Configuration Manager so that each config picks up the tests when the solution is built.

What This Enables

We can select what environment we wish to run against, run the test and see it appear in Browserstack (not in parallel) or run it locally using our own setup.

Bstack_1

So now we have essentially got a single test running in the cloud (and our config still allows us to run locally if we choose), next we need to kick off a bunch of these against different systems.

The Build Script

We use a PowerShell build script to achieve the parallel part of this, here it is:

$solution = "Your.Testing.Solution.sln"

function Get-SolutionConfigurations($solution)
{
        Get-Content $solution |
        Where-Object {$_ -match "(?&lt;config&gt;\w+)\|"} |
        %{ $($Matches['config'])} |
        select -uniq
}

$frameworkDirs = @((Get-ItemProperty -Path "HKLM:\SOFTWARE\Microsoft\MSBuild\ToolsVersions\12.0" -Name "MSBuildToolsPath32")."MSBuildToolsPath32",
                        "$env:windir\Microsoft.NET\Framework\v4.0.30319")
    for ($i = 0; $i -lt $frameworkDirs.Count; $i++) {
        $dir = $frameworkDirs[$i]
        if ($dir -Match "\$\(Registry:HKEY_LOCAL_MACHINE(.*?)@(.*)\)") {
            $key = "HKLM:" + $matches[1]
            $name = $matches[2]
            $dir = (Get-ItemProperty -Path $key -Name $name).$name
            $frameworkDirs[$i] = $dir
        }
    }

    $env:path = ($frameworkDirs -join ";") + ";$env:path"

@(Get-SolutionConfigurations $solution) | foreach {
      Write-Host "Building for $_"
    msbuild $solution /p:Configuration=$_ /nologo /verbosity:quiet
}
 
 New-Item "$(get-location)\packages\specflow.1.9.0\tools\specflow.exe.config" -type file -force -value "&lt;?xml version=""1.0"" encoding=""utf-8"" ?&gt; &lt;configuration&gt; &lt;startup&gt; &lt;supportedRuntime version=""v4.0.30319"" /&gt; &lt;/startup&gt; &lt;/configuration&gt;" | Out-Null

@(Get-SolutionConfigurations $solution)| foreach {
    Start-Job -ScriptBlock {
        param($configuration, $basePath)

        try
        {
            &amp; $basePath\packages\NUnit.Runners.2.6.4\tools\nunit-console.exe /labels /out=$basePath\nunit_$configuration.txt /xml:$basePath\nunit_$configuration.xml /nologo /config:$configuration "$basePath/Your.Testing.Solution/bin/$configuration/Your.Testing.Solution.dll"
        }
        finally
        {
            &amp; $basePath\packages\specflow.1.9.0\tools\specflow.exe nunitexecutionreport "$basePath\Your.Testing.Solution\Your.Testing.Solution.csproj" /out:$basePath\specresult_$configuration.html /xmlTestResult:$basePath\nunit_$configuration.xml /testOutput:nunit_$configuration.txt
        }

    } -ArgumentList $_, $(get-location)
}
Get-Job | Wait-Job
Get-Job | Receive-Job

It obviously needs tweaking in areas to point to your solution but what the hell is it actually doing? It is running msbuild against each config file and executing the test suite. Our Safari based config gets built and executed against BrowserStack as do any other configurations. A test report for each config/ system is then generated to let you know the outcome of that particular run.

Running the script should allow you to see parallel test execution against your desired systems in Browserstack:

Bstack_3

Generated Feedback and Reporting

Reports generated after look similar to the below:

Bstack_2

What Next?

You will undoubtedly encounter issues with Browserstack seeing your internal test environments, this is something that you will need to consider. Consult the Browserstack documentation online regarding tunnelling or running Browserstack locally.

We can also throw in some automated visual checking to really get the ball rolling with Continuous Delivery. If you have some well set up Applitools Eyes base images then visual checking on many systems at once is potentially worth thousands of checks.



Data Driven Specflow Testing

What is it?
Specflow has a plugin called Specflow + Excel to allow you to hoard data in a spreadsheet and run tests using the data contained within.

Example

  • Download Specflow + Excel from Nuget into your solution.
  • Write a normal looking Scenario Outline such as below and notice that there are no items in the example table and also notice the @Tag above the examples table; this is referencing the file we want to use that houses all of our data:
Feature: GoogleTest
In order to have data driven tests in specflow
As a tester
I want to be able to plug all of my data into a spreadsheet

@mytag
Scenario Outline: GoogleTest
Given I have navigated to Google
And I have entered  into the search field
When I press enter
Then the result containing  are shown on screen

@source:GoogleTest.xlsx
Examples:
| SearchTerm | Result |
  • Write out your Step Definitions as usual, no special changes or requirements are needed
  • Create a new Excel Spreadsheet called ‘GoogleTest.xlsx’ and include this in your solution
  • Enter your corresponding parameter values
SearchTerm Result
fish Official Fish site
foo bar foobar2000
specflow SpecFlow – Cucumber for .NET

These will now appear as tests in your test runner!

Specflow Excel




top