BDD frameworks for NET

When I work in a project that implies Business Logic and Behaviors I usually prefer to write my unit tests using a Behavioral approach, because in this way I can easily projects my acceptance criteria into readable piece of code.

Right now I am already on Visual Studio 2013 and I am struggling a bit with the internal test runner UX, because it doesn’t really fit the behavioral frameworks I am used to work with.

So I decided to try out some behavioral frameworks with Visual Studio 2013. I have a simple piece of functionality that I want to test. When you create a new Order object, if the Order Id is not provided with the right format, the class Order will throw an ArgumentException.

If I want to translate this into a more readable BDD sentence, I would say:

Given that an Order should be created
When I create a new Order
Then the Order should be created only if the Order Id has a valid format
Otherwise I should get an error\

So let’s see now the different ways of writing this test …

XBehave

XBehave is probably the first framework I tried for behavioral driven tests. The implementation is easy but in order to get it working properly you have to write your tests on top of xUnit test framework, and of course you have also to install the xUnit test runner plugin for Visual Studio.

In order to write a test, you need to provide the Given, When and Then implementations. Each step has a description and a delegate that contains the code to be tested. Finally, each method must be decorated with a [Scenario] attribute.

[Scenario]
[Example("ABC-12345-ABC")]
public void CreatingAnOrderWithValidData(string orderId, Order order)
{
    _
        .When("creating a new Order",
                   () => order = Order.Create(orderId))            
        .Then("the Order should be created",
                   () => order.Should().NotBeNull());
}

In this specific case I am using the [Example] attribute to provide dynamic data to my test, so that I can test multiple scenarios in one shot.

image

The problem? For each step of my test, Visual Studio test runner identifies a unique test, which is not really true and quite unreadable because this is one method, one test but in Visual Studio I see a test row for each delegate passed to XBehave.

NSpec

NSpec is another behavioral framework that allows you to write human readable tests with C#. The installation is easy, you just need to download the Nuget package, the only problem is the interaction with Visual Studio. In order to run the tests you need to call the test runner of NSpec, which is not integrated into the test runner of Visual Studio, so you need to read the test results on a console window, without the availability to interact with the failed tests.

In order to test your code you need to provide the Given,When and Then like with any other behavioral framework. The difference is that this framework is making an extensive use of magic strings and it assumes the developer will do the same.

void given_an_order_is_created_with_valid_data()
{
    before = () => order = Order.Create("xxx-11111-xxx");
    it["the order should be created"] = 
         () => order.should_not_be_null();
}

And this is the result you will get in Visual Studio Nuget package console (as you can see, not really intuitive and easy to manage):

image 

Honestly the test syntax is quite nice and readable but I found annoying and silly that I have to run my tests over a console screen, come’n the interaction with Visual Studio UI is quite well structured and this framework should at least prints the output of the tests into the test runner window.

StoryQ

I came to StoryQ just few days ago, I was googling about “fluent behavioral framework”. The project is still on www.codeplex.com but it looks pretty inactive, so take it as is. There are few bugs and code constraints that force you to implement a very thigh code syntax, thigh to the StoryQ conventions.

The syntax is absolutely fluent and it’s very readable, compared to other behavioral frameworks, I guess this is the nearest to the developer style.

new StoryQ.Story("Create Order")
.InOrderTo("Create a valid Order")
.AsA("User")
.IWant("A valid Order object")

.WithScenario("Valid order id")
    .Given(IHaveAValidOrderId)
        .And(OrderIsInitialized)
    .When(ANewOrderIsCreated)
    .Then(TheOrderShouldNotBeNull)

.ExecuteWithReport(MethodBase.GetCurrentMethod());

And the output is fully integrated into Visual Studio test runner, just include the previous code into a test method of your preferred unit test framework.

image

The output is exactly what we need, a clear BDD output style with a reference to each step status (succeed, failed, exception …)

BDDify

BBDify is the lightest framework of the one I tried out and honestly is also the most readable and easy to implement. You can decide to use the Fluent API or the standard conventions. With both implementations you can easily build a dictionary of steps that can be recycled over and over.

In my case I have created a simple Story in BDDfy and represented the story using C# and no magic strings:

[Fact]
public void OrderIsCreatedWithValidId()
{
    this.
        Given(s => s.OrderIdIsAvailable())
        .When(s => s.CreateANewOrder())
        .Then(s => s.OrderShouldNotBeNull())
        .BDDfy("Create a valid order");
}

And the test output really fit into Visual Studio test runner, plus I can also retrieve in the test output my User Story description, so that my QAs can easily interact with the tests results:

image

I really like this layout style because I can already picture this layout into my TFS reports.

I hope you enjoyed!