Automated Tester


This content shows Simple View


Pickles Reports – Living Document Generation

Adding Pickles to your Project is really useful for presenting your Gherkin based tests in an easy to read, searchable format with some funky metrics added in. If you get your CI build to publish the output as an artifact then on each build you will always have up to date current documentation of what is being tested.

You can configure it numerous ways, via MSBuild, PowerShell, GUI or Command Line.

In this article I will be setting up via command line and running a batch file as a build step in TeamCity. First off we need to Install Pickles Command Line via NuGet. NUnit 3 is my test runner.

Once built we can start making use of it by running the executable with our desired parameters:

  • –feature-directory: Where your Feature Files live, relative to the executable
  • –output-directory: Where you want your Pickles report to generate
  • –documentation-format: Documentation format (DHTML, HTML, Word, Excel or JSON)
  • –link-results-file: Path to NUnit Results.xml (this will allows graphs and metrics)
  • –test-results-format: nunit, nunit3, xunit, xunit2, mstest, cucumberjson, specrun, vstest

Together it looks like the below, which is put into a batch file and called in a closing build step.

.\packages\Pickles.CommandLine.2.5.0\tools\pickles.exe –feature-directory=..\..\Automation\Tests\Project.Acceptance.Tests\Features^ –output-directory=Documentation^ –link-results-file=..\..\..\TestResult.xml –test-results-format=nunit –documentation-format=dhtml

I use the NUnit format over NUnit3 because I have set my NUnit3 runner to output NUnit2 formatted results, this is so Specflow can consume the same output and produce more metrics on the build. With the results file hooked in you can get some whizzy graphics:


Pickles is a great tool for BDD and should  help bridge that gap between “the Business” and IT. A full sample report can be found here. You can find extensive documentation here for the various ways to set up Pickles.


Data Driven Specflow Testing

What is it?
Specflow has a plugin called Specflow + Excel to allow you to hoard data in a spreadsheet and run tests using the data contained within.


  • Download Specflow + Excel from Nuget into your solution.
  • Write a normal looking Scenario Outline such as below and notice that there are no items in the example table and also notice the @Tag above the examples table; this is referencing the file we want to use that houses all of our data:
Feature: GoogleTest
In order to have data driven tests in specflow
As a tester
I want to be able to plug all of my data into a spreadsheet

Scenario Outline: GoogleTest
Given I have navigated to Google
And I have entered  into the search field
When I press enter
Then the result containing  are shown on screen

| SearchTerm | Result |
  • Write out your Step Definitions as usual, no special changes or requirements are needed
  • Create a new Excel Spreadsheet called ‘GoogleTest.xlsx’ and include this in your solution
  • Enter your corresponding parameter values
SearchTerm Result
fish Official Fish site
foo bar foobar2000
specflow SpecFlow – Cucumber for .NET

These will now appear as tests in your test runner!

Specflow Excel

Gherkin DSL Overview

Gherkin is a Domain Specific Language for bridging the communication gap between “The Business” and “IT”.

Feature for the software are written in English (or any other language for that matter) and then translated into actionable tests using a simple Given, When, But and Then. It has a simple format to follow, let’s have a look with an example.

Feature: Login
 In order to make use of my account
 As a user
 I want to be able to log into my account

Gherkin feature files always start with Feature: “Title of your feature” followed by a few lines detailing what the feature is and what it is doing. This way we know what we are testing.

       Given I have navigated to 'WebsiteName'
       When I click the link 'Login'

Background is not mandatory but is useful if you have some common steps, this gets read or executed before the main guts of the test.

Scenario: Login link goes to login page
       Then I am taken to the Login Page

The main guts of the test: Scenario defines what exactly is being tested followed by more Given, When, But & Then steps to ultimately come to an assert-able outcome.

We can also substitute English words for parameters in which we can pass in many different words. This is called a Scenario Outline and the parameters are defined with Examples in a table format.

Scenario Outline: Invalid Login Credentials
       And I am taken to the Login Page
       When I input my username as 
       And I input my password as 
       Then I should see the error 

       | Username             | Password | Error                                   |                                             
       | | abcdefg  | Password or email address is incorrect. |