Automated Tester

.science

This content shows Simple View

Specflow

Concurrent Test Running with Specflow and NUnit 3

A little while back I wrote a post on achieving concurrent or parallel test running with Selenium, Specflow and NUnit2, but what about NUnit3? Let’s have a look at that, thankfully it is a bit simpler than running an external build script as done previously.

First up, according to the Specflow docs – open up your AssemblyInfo.cs file in your project and add the following line:

[assembly: Parallelizable(ParallelScope.Fixtures)]

Next we need to setup/ teardown our browser and config from the Scenario Hook level, doing it in a Test Run Hook will be problematic as it is static. Your Hooks might look similar to this:

[Binding]
public class ScenarioHooks
{
private readonly IObjectContainer objectContainer;
private IWebDriver Driver;

public ScenarioHooks(IObjectContainer objectContainer)
{
this.objectContainer = objectContainer;
}

[BeforeScenario]
public void BeforeScenario()
{
var customCapabilitiesIE = new DesiredCapabilities();
customCapabilitiesIE.SetCapability(CapabilityType.BrowserName, "internet explorer");
customCapabilitiesIE.SetCapability(CapabilityType.Platform, new Platform(PlatformType.Windows));
customCapabilitiesIE.SetCapability("webdriver.ie.driver", @"C:\tmp\webdriver\iedriver\iedriver_3.0.0_Win32bit.exe");

Driver = new RemoteWebDriver(new Uri(XXX.XXX.XXX.XXX), customCapabilitiesIE);
objectContainer.RegisterInstanceAs<IWebDriver>(Driver);
}

[AfterScenario]
public void AfterScenario()
{
Driver.Dispose();
Driver.Quit();
}
}

You can see from the browser instantiation we are sending the tests to a Selenium Grid Hub, so as a precursor to running the tests you will need suitable infrastructure to run a grid, or you could configure it to go off to SauceLabs or BrowserStack.

Assuming the hub and nodes are configured correctly, when your build process runs the tests then the hub will farm them out by feature file (for other options see the parallel scope in AssemblyInfo.cs) to achieve concurrent test running, and that’s it! Much nicer.



Testing Rest Endpoints with RestSharp and Specflow

In an earlier post I set how to use Specflow to test SOAP endpoints but what about REST API’s?

Given the nature of building URLs to interact with them it should be a cinch!… And it was. By leveraging use of the RestSharp library I was easily able to manipulate an example Rest interface over at http://www.thomas-bayer.com/sqlrest/.

So lets have a look at how:

Each method will be calling on the endpoint we initially set (the base URL), so lets wire in some tests with Gherkin and define some step definitions.

Scenario Outline: RestSharp Get Customer Record
Given I have an example endpoint http://www.thomas-bayer.com/sqlrest/
When I search for customer record 0
Then the result contains customer <ID>
Then the result contains customer <First Name>
Then the result contains customer <Last Name>
Then the result contains customer <Street>
Then the result contains customer <City>

Examples:
| ID | First Name | Last Name | Street | City |
| 0 | Laura | Steel | 429 Seventh Av. | Dallas |

Scenario: Post Product Price
Given I have an example endpoint http://www.thomas-bayer.com/sqlrest/
And I update the price of Product 1 to 10.00
When I search for Product 1
Then the price is 10.00
And I reset the price of product 1 to 24.8

We are passing the endpoint down to our helper here but I would normally put it in the App.config to keep things configurable.

Next up I created a Helper called RestHelper which will deal with all the setup, GET and POST requests etc. We need to define the endpoint and assign it to a commonly accessible place.

public class RestHelper
{
public RestClient endpoint = null;

public RestClient SetEndpoint(string endpointUrl)
{
endpoint = new RestClient(endpointUrl);
return endpoint;
}

Note: Your endpoints will most likely require some authentication of some kind which I am not covering. However I was easily able to add cookie authentication with a little tweaking.

Next lets create some other useful helper methods like GET and POST:

public string GetQuery(string query)
{
var request = new RestRequest(query, Method.GET);
IRestResponse response = endpoint.Execute(request);
var content = response.Content; // raw content as string
return content;
}

My POST method is simply to update the price of a product but you could rework to whatever you need:

public void UpdatePrice(string query, string price)
{
var request = new RestRequest(query, Method.POST) { RequestFormat = DataFormat.Xml };
var body = ("<resource><PRICE>" + price + "</PRICE></resource>");
request.AddParameter("text/xml", body, ParameterType.RequestBody);
endpoint.Execute(request);
}

In our Step definitions we just need to new up the Helper and pass in the right query, we are also assigning the Rest query result in a place that can be accessed by multiple step definitions:

private readonly RestHelper Rest = new RestHelper();

private string queryResult = null;

[Given(@"I have an example endpoint (.*)")]
public void GivenIHaveAnExampleEndpoint(string restEndpoint)
{
Rest.SetEndpoint(restEndpoint);
}

[When(@"I search for customer record (.*)")]
public void WhenISearchForCustomerRecord(string customerNo)
{
queryResult = Rest.GetQuery("CUSTOMER/" + customerNo + "/");
}

[Given(@"I update the price of Product (.*) to (.*)")]
[Then(@"I reset the price of product (.*) to (.*)")]
public void GivenIUpdateThePriceOfProductTo(int productNo, string newPrice)
{
Rest.UpdatePrice("PRODUCT/" + productNo +"/", newPrice);
}

And so on… If you want to have a play around with the code you can clone it from here and have fun testing REST APIs!



Specflow Reports with NUnit3

This post will look at setting up Specflow reports to work with NUnit3. When you install Specflow via NuGet, there is a tools folder which contains specflow.exe – if we run a batch process after the build on the CI server which points to the NUnit3 TestResult.xml, then we can generate a report and reference it as an artefact in TeamCity (and similar CI software).

But there is a catch, which isn’t (at the time of writing) explained in the Specflow reporting documentation. Currently Specflow (v 2.0.0) cannot interpret the NUnit3 TestResult output, so to counter this restriction we need to configure NUnit3 to output its TestResult.xml in the NUnit2 format. A full list of NUnit3 console options can be found here.

nunit3-console.exe –where cat==SmokeTests&&cat!=ExcludeFromCI
–out=TestResult.txt
–result=TestResult.xml;format=nunit2 C:\Path\To\Acceptance.Tests.dll

With this in place it’s now just a case of rigging up the Specflow command to point to the Test Result data and telling it to produce a report and specifying where to output the HTML report.

specflow.exe nunitexecutionreport Your.AcceptanceTest.csproj /testResult:TestResult.xml
 /out:MyResult.html

Once you have the TestResult.xml and the Result output generating where you want, it’s easy to wrap the command up into a batch file and call it from a build step after your tests have executed. Make this output an artefact and then you have metrics in place for your nightly and/ or other test builds.

Demo Specflow Report



Recording Videos of your Selenium Tests

It is good practice to take a screen grab of test failures and review them later, particularly if you have some bizarre wait or timeout error that can be difficult to pin down but sometimes even these can lead you down the wrong path or not give enough context to what went wrong. This led me to explore being able to capture videos of running tests that can be scrutinised at a later time, much like you get when you run tests in SauceLabs or BrowserStack.

I created a helper class called RecordVideo and pulled down a NuGet package called Microsoft Expression Encoder. You will also need to install Microsoft Expression Encoder 4.

Having set up a start (and stop) recording method we can easily capture tests in and write them out to file with the help of our set up or specflow hooks.

Here’s how:

using System;
using Microsoft.Expression.Encoder.ScreenCapture;

public class RecordVideo
{
private static readonly ScreenCaptureJob vidRec = new ScreenCaptureJob();

public static void StartRecordingVideo(string scenarioTitle)
{
string timestamp = DateTime.Now.ToString("dd-MM-yyyy-hh-mm-ss");
vidRec.OutputScreenCaptureFileName = @"C:\Test Videos" + scenarioTitle + " " + timestamp + ".wmv";
vidRec.Start();
}

public static void EndRecording()
{
vidRec.Stop();
vidRec.Dispose();
}
}

We can now call these in our setup/ teardown (in my case I’m using Scenario Hooks) and use the test case name for the file title along with a timestamp.

[BeforeScenario]
public void BeforeScenario()
{
RecordVideo.StartRecordingVideo(ScenarioContext.Current.ScenarioInfo.Title);
objectContainer.RegisterInstanceAs(new PageContext(TestRunContext.Driver));
}
[AfterScenario]
public void AfterScenario()
{
if (ScenarioContext.Current.TestError != null)
{
TakeScreenShot.SaveScreenShot(TestRunContext.Driver, TestFailurePath + ScenarioContext.Current.ScenarioInfo.Title + ".jpeg");
}

RecordVideo.EndRecording();
}

Be sure to put something in place to clear the directory out of videos on each run or you might find yourself running out of disk space quite quickly.



API Testing with Specflow

I’m a big fan of Specflow/ Cucumber, it’s great for creating tests to run against a front end with webdriver but what if we want to keep that same plain English BDD approach and apply it against an API for automated API Testing? It’s certainly possible and this article will cover the basics to get you up and running.

I am going to assume you have a framework set up and in place, you can even use a webdriver based solution as described previously, although you may wish to run it with a headless driver.

I’m also going to add a SOAP endpoint to my solution and write a test in Gherkin, this endpoint is the publicly available Global Weather SOAP endpoint available here:

http://www.webservicex.com/globalweather.asmx?wsdl

This WSDL generates 2 service operations, GetCitiesByCountry (to get a list of all the cities for any country) and GetWeather (to get weather for any city/country combination).

Let’s add the WSDL somewhere we can use it in Visual Studio. If you right click your project and go to Add > New > Service Reference, you will be presented with a dialogue in which you can paste in the WSDL, click Go and give it an appropriate name.

Add Service Reference for API Testing

Click OK, you should see a new Service References folder in your solution. Lets begin formulating a test for our API Testing suite.

@API
Scenario Outline: Get Cities
Given I query the API
Then I can retrieve <City> from <Country>

Examples:
| City        | Country        |
| Reims       | France         |
| London      | United Kingdom |
| Agra        | India          |
| Porto Santo | Portugal       |
| Cancun      | Mexico         |
| Pretoria    | South Africa   |
| Melbourne   | Australia      |

We’re going to need a helper class which I will call ‘GeoHelper’ where we can reference the endpoint and then make some methods we can call from the step definition. Below I am passing a string value to the ‘GetCitiesByCountry’ extension on the endpoint and returning the result. Then I have created another helper method which takes that result and asserts the city is present in the response.

public class GeoHelper
{
     private readonly GlobalWeatherSoap globalWeather = new GlobalWeatherSoapClient();

     public string GetCitiesByCountry(string country)
     {
          var x = globalWeather.GetCitiesByCountry(country);
          return x;
     }

     public void CheckCityNameIsPresent(string countryName, string cityName)
     {
          Thread.Sleep(200);
          Assert.IsTrue(GetCitiesByCountry(countryName).Contains(cityName));
     }
}

So our step definition calls this like so:

private readonly GeoHelper geo = new GeoHelper();

[Then(@"I can retrieve (.*) from (.*)")]
public void ThenICanRetrieveFrom(string cityName, string countryName)
{
     geo.CheckCityNameIsPresent(countryName, cityName);
}

And that in brief is how we can leverage the use of Specflow for API Testing.




top