Continuous Testing

Various software vendors have produced some useful Visual Studio tooling to help TDD and BDD for developers.

They range from automatic build and running of affected tests on save to showing you the location of failing tests and code coverage figures within the coding IDE.

If you would like a demo project with tests to try out some of these tools you can download the following solution:

http://stevenhollidge.com/blog-source-code/LibrarySystem.zip

And, (for me) the winner is…

NCrunch for Visual Studio 2010

NCrunch is an automated parallel continuous testing tool for Visual Studio .NET. It intelligently takes responsibility for running automated tests so that you don't have to, and it gives you a huge amount of useful information about your tests (such as code coverage and performance metrics) inline in your IDE while you work.

Here are the rest:

Continuous Testing for Visual Studio 2010

• MSTest for Visual Studio 2010
• NUnit v2.5 or later
• XUnit v1.5 or later

Continuous Testing for Visual Studio auto-detects your unit tests and runs them each time you build your solution. It adds an error to your error list for each test that fails, allowing you to navigate to the line of the test that failed, just like you would navigate from a compile error.

Continuous Testing removes a manual step for you, making your workflow far more efficient. 

There is a free version and a professional paid version with various enhancements such as auto ordering tests and aborting a test run on a first test failure.

Link: http://visualstudiogallery.msdn.microsoft.com/c074d3c6-71e2-4628-9e7c-7690e706aef4

Demon from RedGate

Demon takes over the Visual Studio build to compile and run tests only for code that has changed.  See the nice green bar down the left side gutter (to the right of the collapse +/-)?  That’s what Demon adds to show your changes are all good.  If your changes result in tests breaking it changes to red.  It’s still in beta and doesn’t seem to be successful all the time but hopefully by the time it comes out of beta this may well become a must have tool.

Link:  http://www.red-gate.com/products/dotnet-development/dotnet-demon/

Mighty Moose from Continuous Tests

Again, this tool runs tests on build only for code that has changed.  You also get code coverage numbers in the left hand side gutter to give you a heads up on how many tests cover the overall method.

Link:  http://continuoustests.com/

And, here’s some old school tooling

Here is a few quick screenshot reminders for anyone that hasn’t already used ReShaper, TestDriven.NET and NCover tools:

ReSharper from JetBrains

You get a nice visual test runner with ReSharper, plus the nice icons next to each test in the coding IDE to allow individual running or debugging of tests or test cases.  Test cases are where you use the same test with multiple attributes above the method signature to pass in different test cases.

Coverage

NCover

Link: http://www.ncover.com

Performance

TryExecute with timeout and maxAttempts

A nice piece of code for calling services with a timeout and maximum attempts, in case the service should cause an exception.

It follows the standard practise for "Try" in .Net, like TryParse, of returning a boolean to indicate success and the result as an out parameter. Should you wish to find out the exception that may have occurred this is also passed as an out parameter.

I've also included an Execute mthod that will raise an exception should the service request timeout or fail more than the maximum number of attempts,

The source code solution includes unit tests in MBUnit, NUnit, xUnit and MSTest:

http://stevenhollidge.com/blog-source-code/TryExecute.zip

QUnit: TDD with JavaScript

QUnit is a TDD framework that allows developers to write unit tests against their JavaScript libraries and run them in the browser or against a CI server. 

For this blog article I’ll be showing you how to setup your dev machine to write and run the tests.

What shall we Unit Test today?

For this example I am going to create some unit tests around a popular jQuery pub/sub plugin, which is available here: https://github.com/phiggins42/bloody-jquery-plugins

The plugin API is very simple and allows the developer to: 

• subscribe to an topic
• unsubscribe from a topic
• publish a topic

My Folder and File Layout for QUnit

When you download the source code to my example you’ll see the following files:

core	Folder	Contains the two core libraries to run QUnit:  QUnit.js and QUnit.css (JQuery is also required, online file used)
source	Folder	Contains the JavaScript you will be testing
tests	Folder	The unit tests
TestRunner.html	Folder	Open in the browser to run the tests

This layout is purely my preference but helps me keep a nice structure to my tests and CI build.

The Test Runner

The Tests

The first couple of lines are just const variables for topics and can be ignored. 

module("Pub/Sub Tests"); gives the tests a grouping (see test output image at the top of the blog article.

test("test name”, function () { … }); is the unit test name and code.

expect(1); states the number of expected assertions within the test.  If this number is not realised the test will fail.

ok(error === null, 'No error occurred'); and equal(count, 2, 'callback was fired twice'); are examples of assertions.

Links

My example source code: http://stevenhollidge.com/blog-source-code/QUnit.zip

QUnit source: https://github.com/jquery/qunit

QUnit docs: http://docs.jquery.com/Qunit

MSpec

You can add MSpec to your project using Nuget or from GitHub:

https://github.com/machine/machine.specifications

After showing examples for BDD with StoryQ and SpecFlow, here is my favourite MSpec.

Here is the test code for our MSpec tests:

Source code:

http://stevenhollidge.com/blog-source-code/BDD-REST-Demo.zip

SpecFlow BDD

You can add SpecFlow to your project using Nuget or you can download from:  http://specflow.org 

After using StoryQ in my previous post I thought I’d show the steps to produce the equivalent tests with SpecFlow.

Once you’ve installed SpecFlow to your machine and added the NUnit.Framework and SpecFlow dlls to your project you can create a SpecFlow Feature file:

The tests won’t pass yet because you need to bind the Step definitions to C# code:

Create a new C# class file and copy and paste the binding code above into it:

We now get running tests that have yet to be implemented:

So we can now fill in the implementation for each of the steps, adding in our BeforeScenario (setup) and AfterScenario (teardown) methods:

To download the source:

http://stevenhollidge.com/blog-source-code/BDD-REST-Demo.zip

StoryQ BDD

You can download StoryQ from CodePlex:  http://storyq.codeplex.com/

StoryQ allows developers to write runnable system requirements in the Gerkin language format (Given, When, Then).

It comes with a nice little tool that converts GWT text into C# code for StoryQ using a fluent interface.

We can then turn up the code generator to produce stubs for our test class.

PLEASE NOTE: When you cut and paste this code you must add your own namespace, otherwise you will get errors later on using StoryQ)

When I paste this code into a C# class and run the test, MSTest reports these steps as pending, due to NotYetImplemented errors:

Here’s the updated code with:

• Implementations added
• A bit of refactoring
• An additional scenario added
• Execute changed to ExecuteWithReport (see below for HTML report)

We can see the initial scenario passed and the invalid credentials failed.  This is because I haven’t coded up the validation of the username and password within the service yet.  And the development cycle continues…

HTML Report Output

As we used the ExecuteWithReport() method we can also take a look in the bin folder and see a StoryQ_Report folder:

Within that report you can find a XML file which you can open in Internet Explorer.

Source code can be found here:

http://stevenhollidge.com/blog-source-code/BDD-REST-Demo.zip