Automated Tester

.science

This content shows Simple View

Load Testing

Measuring Speed and Performance with Sitespeed.io

I recently came across a fantastic open source speed and performance tool called Sitespeed and was pretty amazed by both what it can do and how it presents the results to you.

It has its foundations in some other cool stuff you may have heard of like YSlow, PhantomJS and Bootstrap and makes good use of WebPageTest too. It seems to support pretty much every platform too, I’m a .Net/ windows guy so I’ll be working with that. Lets have a look:

Install

Install via node – if you don’t have NodeJs get it here.

$ npm install -g sitespeed.io

Configure and Run

So to get some metrics out of something, lets run a simple test. Open a command prompt and run the following:

sitespeed.io.cmd -u http://www.google.co.uk -d 0 -b chrome -v –viewPort 1920×1080 –name Google_Report

If you’re a windows user like me don’t forget the .cmd on the end of the sitespeed.io initialiser, it’s an easy mistake to make! Let Sitespeed.io do it’s thing, in this particular instance we are crawling google to a depth of 0 (so just it’s landing page, for brevity.) and doing this with chrome, viewport size is set to 1920×1080, -v for verbose stdout, name gives it a name.

Running sitespeed.io

Hey presto, a performance dashboard:

Sitespeed.io report

Full example report is here.

We’re not really measuring against anything here just getting something pretty to look at, sitespeed.io recommends we create a Performance Budget to measure against which sounds like good advice to me, if you don’t know much about it they even recommend a some great articles to broaden your mind and follow best practice, to quote their site:

“Have you heard of a performance budget? If not, please read the excellent posts by Tim Kadlec Setting a performance budget and Fast enough. Also read Daniel Malls How to make a performance budget. After that, continue setup sitespeed.io :)”

Add this into your parameters and your report will illustrate what is falling in and out of your budget. So what else can we do with sitespeed.io?

  • Add a list of URLs to test against
  • Add authentication if you need to login somewhere (–requestHeaders cookie.json for example)
  • Drill down and get HAR files, maybe even convert to jmx for stress testing
  • Get screenshots
  • Get results in different formats (HTML/ XML and such)
  • Do multiple passes for comparison
  • Compare multiple sites
  • Throttle connections
  • Supports Selenium for running with different browsers
  • Supports Google Page Speed Insights
  • Supports Docker

In fact, there is way too much to list out here – luckily sitespeed.io has some very good documentation on how to configure so it should be easy to fine tune it to your needs. Furthermore if you use some other cool stuff like Graphite you can output the results to there.

Summary

The benefits of this should be clear, we can hook up a command in a build step on a CI server to either output some funky looking graphite metrics or generate performance dashboards regularly to catch performance issues before they get released.

I don’t yet know if this tool can be incorporated into existing automation tests with Selenium… or really if you would even want to do that, I had been looking into using Jan Odvarko’s HAR Export Trigger to compare against YSlow scores but sitespeed.io seems to kill lots of birds with one stone.

My first impressions are that sitespeed.io is a very powerful, easy to configure, cross platform performance tool that displays data in such as way that is easy to understand.

Bootnote

Keep your eyes peeled for sitespeed.io 4.0 which will have even more cool stuff going on, due to be released in a few weeks time.



Load Testing made easy with BlazeMeter & JMeter

Continuing for a moment with load testing lets have a look at Blaze Meter…

What the tools are

  • BlazeMeter: A Chrome App that allows you to record journeys through a system and export a load test plan to JMX Format. You will need to sign up for an account. BlazeMeter.com
  • JMeter: A load testing tool that can run files in JMX format. JMeter.Apache.org

How to create a load test plan

If installed correctly you should see the following logo in Chrome BlazeMeter

When clicked the BlazeMeter panel is displayed, allowing you to input a test plan name and change a few other basic parameters such as the type of system doing the load test and the origin of the traffic.

BlazeMeter_Panel

All you need to do is hit record and perform the actions you wish to load test.

When you have completed this you can export the captured JSON into a JMX file and download it by clicking the .JMX button on the panel.

Open and edit with JMeter

Open your newly downloaded test plan in JMeter, change the threads/ ramp up and loop count to whatever is desired.

Next add the appropriate listeners to your plan and hit Run. Job done.

BlazeMeter_JMX



Load Testing with Selenium and BrowserMob Proxy

In days of yore, Selenium had the capability to capture network traffic and manipulate it. This was taken out of later versions as the mantra for the project was to mimic user actions.

We can still harness this capability and do some other cool things along the way such as blacklisting or whitelisting certain services, simulate slow connections, write data and produce metrics.

How To

We can only harness this capability by spinning up our tests via a proxy and that is where our first tool comes in; BrowserMob Proxy. There is also a write up on it here and which coincidentally I lifted some of the code from: Ada The Dev – BrowserMob Proxy, the blog of one of the .Net Gurus of BrowserMob.

When we are setting up our start up method we need to instantiate the proxy and then spin up the browser.

// Supply the path to the Browsermob Proxy batch file
            Server server = new Server(@"C:\Users\user.name\Desktop\BMP\bin\browsermob-proxy.bat");
            server.Start();
            Client client = server.CreateProxy();
            client.NewHar("Load Test Numbers");
            var seleniumProxy = new Proxy { HttpProxy = client.SeleniumProxy };
            var profile = new FirefoxProfile();
            profile.SetProxyPreferences(seleniumProxy);

            // Navigate to the page to retrieve performance stats for
            Driver = new FirefoxDriver(profile);
            Driver.Navigate().GoToUrl("http://www.google.co.uk");

After we have started the proxy we are instructing the client to create a new HAR file called “Load Test Numbers”. A HAR file is a Http Archive. More information can be found on that here: Har File Spec v1.2.

After navigating to Google we need to get the performance statistics and then view them in some way. This is easy to do in debug mode, looking into variables or writing the content out to the console.

// Get the performance stats
         HarResult harData = client.GetHar();

 // Do whatever you want with the metrics here. Easy to persist
 Log log = harData.Log;
            Entry[] entries = log.Entries;

            var file = new System.IO.StreamWriter("c:\\test.txt");
           
            foreach (var entry in entries)
            {
                Request request = entry.Request;
                Response response = entry.Response;
                var url = request.Url;
                var time = entry.Time;
                var status = response.Status;
                Console.WriteLine("Url: " + url + " - Time: " + time + " Response: " + status);

                file.WriteLine("Url: " + url + " - Time: " + time + " Response: " + status);
            }

            file.Close();

In the code above we a getting select elements from the HAR (URL, Time and Status Code) and writing them out to the console and also a text file.

Viewing the entire HAR

We can capture the JSON generated in the HAR file and write out the HAR to disk. Using this HAR we can use another tool to view it and it’s metrics. We can also convert the HAR to a JMX file which creates for us a JMeter Test plan.

Once we retrieve the performance stats we need to serialise the content:

// Get the performance stats
            HarResult harData = client.GetHar();

            var json = new JavaScriptSerializer().Serialize(harData);
           
            Log log = harData.Log;
            Entry[] entries = log.Entries;

            var file = new System.IO.StreamWriter("c:\\test.har");
            file.WriteLine(json);

This writes a serialised HAR file out to file. If we paste the content into HAR Viewer we magically get a whole bunch of useful metrics:

HAR1

HAR2

Converting to JMX

This is simple to do, as with pasting your HAR data into HAR Viewer. You can paste the same JSON in Flood.IO Har2JMX and download the JMX. Now simply load into JMeter, change the Thread number, ramp up, loops and then add desired listeners and you are ready to roll.

Unfortunately there is currently no way of programatically converting HAR2JMX in C#. The Flood.IO code is open source however.

BlazeMeter_JMX

Exporting a HAR straight from the Browser

To do this you need FireBug and NetExport. Once installed you can save all network traffic logged in FireBug to a HAR format and even view in firebug by pressing F12. If you look under the Net tab. Selecting Export >> Save As will get you your HAR.

HAR3




top