Yesterday on twitter I had the start of an interesting conversation with Steve Harmann and Scott Bellware, but then I had to do work, so I had to stop just when it was getting interesting.

One of the things I have found useful in writing unit tests for an application I work on is mbunit’s CombinatorialTest. If you are familiar with RowTests in mbunit, CombinatorialTests are similar, but allow you to use a factory method to generate the parameters for the test.

By way of example, below is a dumbed down version of a real test I have. Note that this was developed before I really knew anything about BDD.

The system under test needs to find the next best move as a number. There are two candidate moves. If the candidates are on both sides of the number we’re on now (mixed signals), we don’t move. If they are both on the same side, we move that direction, but to the number with the smallest change from our current number. I received these specs from our product owner in the form of an Excel spreadsheet with a formula, mixed with a crap-load of test data.

I have found that it’s useful to define a class for the parameter to the test, and that it’s really useful to have that class have lambdas for the actual test, and that it’s really, really useful to use Expression<> for the lambdas, so you can print the result. I am going to call this class a “Specification Class” for lack of a better term. Here’s the one for the stories described above:

public class PositionSpec

{

    public decimal Candidate1 { get; set; }

    public decimal Candidate2 { get; set; }

    public decimal Current { get; set; }

    public Expression<Predicate<ModelPosition,decimal>> Test { get; set; }

    public override string ToString()

    {

        return String.Format("c1={0:p} ^ c2={1:p} ^ current={2:p} -> {3}", 

            Candidate1, Candidate2, Current, Test.Body.ToString());

    }

}

Notice that the Test property is an Expression. If you are not familiar, this is like a lambda - Expression<Predicate<ModelPosition, decimal>> can do everything Predicate<ModelPosition, decimal> can do – but you can examine the body of the expression. I think this is really cool and a gateway to functional+dynamic programming. In this case though, I just want to use that so I can print the expression in ToString – mbunit will call this when printing the name of the test in the results, so we can see which expression we are testing.

Here’s the factory function that produces instances for tests (specifications):

public IEnumerable<PositionSpec> TestPositions()

{

    yield return new PositionSpec() { Candidate1 = .15M, Candidate2 = .20M, Current = .17M, 

        Test = (pos, w) => w == .17M };

    yield return new PositionSpec() { Candidate1 = .15M, Candidate2 = .20M, Current = .10M, 

        Test = (pos, w) => w == .15M };

    yield return new PositionSpec() { Candidate1 = .15M, Candidate2 = .10M, Current = .20M, 

        Test = (pos, w) => w == .15M };

    yield return new PositionSpec() { Candidate1 = .15M, Candidate2 = .13M, Current = .10M, 

        Test = (pos, w) => w == .13M };

    yield return new PositionSpec() { Candidate1 = .15M, Candidate2 = .17M, Current = .20M, 

        Test = (pos, w) => w == .17M };

    yield return new PositionSpec() { Candidate1 = .15M, Candidate2 = .20M, Current = .17M,

        Test = (pos, w) => pos.IsNoActionRequired == true };

    yield return new PositionSpec() { Candidate1 = .15M, Candidate2 = 0M, Current = .14M,

        Test = (pos, w) => pos.IsNoActionRequired == true };

    yield return new PositionSpec() { Candidate1 = .15M, Candidate2 = 0M, Current = .14M,

        Test = (pos, w) => w == .14M};

    yield return new PositionSpec() { Candidate1 = .15M, Candidate2 = 0M, Current = .20M,

        Test = (pos, w) => w == .15M};

    yield return new PositionSpec() { Candidate1 = 0M, Candidate2 = .15M, Current = .14M,

        Test = (pos, w) => pos.IsNoActionRequired == true };

    yield return new PositionSpec() { Candidate1 = 0M, Candidate2 = .15M, Current = .14M,

        Test = (pos, w) => w == .14M};

    yield return new PositionSpec() { Candidate1 = 0M, Candidate2 = .15M, Current = .20M,

        Test = (pos, w) => w == .15M};

    yield return new PositionSpec() { Candidate1 = 0M, Candidate2 = 0M, Current = .20M,

        Test = (pos, w) => w == 0M};

}

And finally the test that consumes them:

[CombinatorialTest]

public void should_result_in_smallest_delta_from_current_with_netting(

    [UsingFactories("TestPositions")] PositionSpec spec)

{

    Mocks.ReplayAll();

    ModelPosition candidate1 = new ModelPosition(){ Weight = spec.Candidate1 };

    ModelPosition candidate2 = new ModelPosition()

    {

        Weight = spec.Candidate2,

        HeldPositionWeight = spec.Current

    };

    decimal result = _methodology.GetNetPositionWeight(candidate2, candidate1);

    Assert.IsTrue(spec.Test.Compile().Invoke(candidate2, result));

}

The result has the expression bodies printed out, so looks like:

------ Test started: Assembly: XXX.Tests.dll ------

Starting the MbUnit Test Execution

Exploring XXX.Tests, Version=1.0.0.2, Culture=neutral, PublicKeyToken=null

MbUnit 2.41.232.0 Addin

Found 13 tests

[assembly-setup] success

[success] NettingMethodologyTests.BaseSetup.should_result_in_smallest_delta_from_current_with_netting(TestPositions(c1=15.00 % ^ c2=20.00 % ^ current=17.00 % -> (w = 0.17)))

[success] NettingMethodologyTests.BaseSetup.should_result_in_smallest_delta_from_current_with_netting(TestPositions(c1=15.00 % ^ c2=20.00 % ^ current=10.00 % -> (w = 0.15)))

[success] NettingMethodologyTests.BaseSetup.should_result_in_smallest_delta_from_current_with_netting(TestPositions(c1=15.00 % ^ c2=10.00 % ^ current=20.00 % -> (w = 0.15)))

[success] NettingMethodologyTests.BaseSetup.should_result_in_smallest_delta_from_current_with_netting(TestPositions(c1=15.00 % ^ c2=13.00 % ^ current=10.00 % -> (w = 0.13)))

[success] NettingMethodologyTests.BaseSetup.should_result_in_smallest_delta_from_current_with_netting(TestPositions(c1=15.00 % ^ c2=17.00 % ^ current=20.00 % -> (w = 0.17)))

[success] NettingMethodologyTests.BaseSetup.should_result_in_smallest_delta_from_current_with_netting(TestPositions(c1=15.00 % ^ c2=20.00 % ^ current=17.00 % -> (pos.IsNoActionRequired = True)))

[success] NettingMethodologyTests.BaseSetup.should_result_in_smallest_delta_from_current_with_netting(TestPositions(c1=15.00 % ^ c2=0.00 % ^ current=14.00 % -> (pos.IsNoActionRequired = True)))

[success] NettingMethodologyTests.BaseSetup.should_result_in_smallest_delta_from_current_with_netting(TestPositions(c1=15.00 % ^ c2=0.00 % ^ current=14.00 % -> (w = 0.14)))

[success] NettingMethodologyTests.BaseSetup.should_result_in_smallest_delta_from_current_with_netting(TestPositions(c1=15.00 % ^ c2=0.00 % ^ current=20.00 % -> (w = 0.15)))

[success] NettingMethodologyTests.BaseSetup.should_result_in_smallest_delta_from_current_with_netting(TestPositions(c1=0.00 % ^ c2=15.00 % ^ current=14.00 % -> (pos.IsNoActionRequired = True)))

[success] NettingMethodologyTests.BaseSetup.should_result_in_smallest_delta_from_current_with_netting(TestPositions(c1=0.00 % ^ c2=15.00 % ^ current=14.00 % -> (w = 0.14)))

[success] NettingMethodologyTests.BaseSetup.should_result_in_smallest_delta_from_current_with_netting(TestPositions(c1=0.00 % ^ c2=15.00 % ^ current=20.00 % -> (w = 0.15)))

[success] NettingMethodologyTests.BaseSetup.should_result_in_smallest_delta_from_current_with_netting(TestPositions(c1=0.00 % ^ c2=0.00 % ^ current=20.00 % -> (w = 0)))

[reports] generating HTML report

TestResults: file:///C:/Users/cbilson/AppData/Roaming/MbUnit/Reports/XXX.Tests.Tests.html

13 passed, 0 failed, 0 skipped, took 3.30 seconds.

So that’s one example. I had a lot of little tiny contexts. I wanted to apply the same basic proto-spec to each one. I could have done this with a base context class and a bunch of subclasses, which seems to be the BDD way to do this, but using combinatorial tests seems more concise and readable to me.

There’s one other really powerful thing you can do with combinatorial tests. Since you can use these to separate context from spec, you could apply a set of specs to a set of contexts. Your actual tests would be the cross product of the two, without having to specify each product.

For example, say you had a really complicated data entry screen. Your user stories indicated that everyone needed to be able to edit data in some subset of the fields on the screen, only admins another subset, and only under certain conditions everyone could edit all fields.

There are lots of ways to tackle this, but one way would be to define the two different subsets using two factories. Then your specifications could be really simple and readable, like “everyone_should_be_able_to_edit_the_everyone_fields”, “only_admins_can_edit_the_admin_fields”, and “everyone_can_edit_the_admin_fields_in_some_special_case".”

For admin vs. normal user fields, it would probably be better to just tag the fields somehow, but what if there were way more conditionals, like “for gold card members with the super-duper price-protection plan, normal customer service reps should be able to edit the discount field.” I have a couple of cases like this where the user interface changes a lot based on a couple of different special conditions and I have found combinatorial tests really good for this.

Now that I am getting into BDD, I think I could do a better job of separating the context from the specification. It seems like the “specification” class I had above should really be the “context” and I should remove the Test property, since that’s the core of the specification. I could then use this technique when I have a specification that applies to a large number of contexts.

One of the presentations I got to go to at Code Camp PDX was on Mocking, and to my excitement, Rhino Mocks was used for all the examples. The Presenter was John Hann, and I think he did a great job. There is also a lot of useful content on his weblog. If you are test-driven, or trying to get there, check it out!

Long time since I last posted. Need less to say, I have been a little busy. I hope to resume most of my regular life soon, but here are a few highlights (to be followed with further posts later) of what I've been up to:

• Julie and Allie came back from a month long trip to Taiwan, to visit Julie's mom. Allie got her second hair cut - the last was when she was 1. She went from waste length hair to just past the shoulders. She made amazing progress in Chinese. Before this trip, she had a bad accent and very limited vocabulary. Hanging around with two cousins she adores, as well as dozens of other extended family members willing to spend time doing things with and talking to her made a huge difference.
• At the beginning of November, I was invited to participate in a week long training session my company puts on - not for developers, but for analysts. It was a really neat experience and I think it speaks highly of our management - that they get the need to keep developers as up to speed as anyone else on how the business works. I learned a ton of stuff, and go to spend time with top minds in the industry (and bug them with questions.) In addition, I met a bunch of cool people from our offices all over the world. We have a lot of work to do on the technology front to improve our organization, but I have now doubt that we're going to do it. This company will grow amazingly over the coming years, and I am glad to be here to take part in that.
• We were going to go to our friends Monique and Dominique's house, in Portland, for Thanksgiving, but ended up inviting a few people over. More people got invited. Then a few more. In the end, I think 19 people showed up. We had a blast, but it was a lot of work to get ready and clean up.

Enough personal stuff...

• I enjoyed the final day of the Seattle Code Camp. I got a chance to talk to two of my agile heroes, Brad Wilson, and Peter Provost. I also got to talk to Stuart Celarier, and was introduced to the Seattle XP Group. Too bad their meetings are the same day as South Sound .NET - which I haven't gone to either in several months. Argg...
• Speaking of user groups: Some co-workers finally got me excited enough to form an in house Developers Group. I had the good fortune of getting to know someone in our company that is one of those "do-er" kinds of people - the kind that can organize a teleconference on 3 continents with 100 people, in under an hour, without batting an eye. I kind of bashfully asked if there was any way we could use one of our company's fancier meeting/conference rooms after hours for around 20 people. She basically said, "Go for it!" and I found that it was actually as easy as inviting the room to our conference. I could even invite coffee and his friends pizza and sandwiches from the cafeteria downstairs if my boss was willing to pay for them (hmm...haven't asked about that...) We got a sweet conference room on the twelfth floor, with an awesome view of the Puget sound, a massive projector and computer, fancy mood lighting, comfy chairs, a mezzanine level - the whole nine yards. I hope about 10 people will show up, which we will try to grow to 20 or 30 as time passes. Thanks Wei, Michael, and Ariel for finally pushing me over the edge on this!
• I've been learning SSIS. It's been kind of frustrating, kind of fun. Mostly, it's only fun when it works. It was really painful to get to the "works" stage with my first couple of projects, but aside from a few gripes, I see how powerful and useful it could be going forward.
• Our development team at work feels like it's getting tighter. Inspired by Brad and Peter, we spent more energy trying to do pair programming and share code better. I think we are good, but we could be a lot better. We only have one project we are doing right now that is truly owned by all developers on the team. The other 4 projects we actively work on all have code owners, and we are working to abolish that.
• My boss said I was a "hippy coder" at a recent team meeting held down the street, in an...umm...off site meeting facility with fermented beverages. I was a little shocked. I try to be open minded, but compared to most of the other developers in our organization, I think I am _more_ disciplined - I use TDD, automated builds, normalized database schemas, consistent names, etc., in comparison with the local standard, which is to say "good enough" a lot. I guess I was just kind of shocked about this, but my boss has a history of making startling observations about me, which give me great cause for self reflection - which is a good thing.
• I installed Vista...like it...not the stuff of dreams, but lots of little things I like (parental controls especially.)
• I finally checked out the "Baroque Cycle" from the local library. What a read. Lots of history. Lots of math and science. Heavy. Sucked up a lot of time.
• I have been working on something kind of neat, integrating SQL Server, our applications, and OLAP. For a taste, here's a paper a colleague wrote several years ago, about doing almost the same thing, just using SAS for the application part (where I would be using .NET.)

Anyway, I hope to blog more now. Later!

Inspired by Roy and Andrew, I have been itching to try using MbUnit on a new project. This evening I got a chance to do that. I have a tool I use for several projects, and I wanted to add a bunch of features to it, so I seized the day.

It's been at least a year since I used MbUnit for a real project. In that time, my unit testing style has changed a lot.

I'd say the biggest change for me came after watching Brad Wilson write unit tests in front of an audience at Seattle Code Camp last year (link broken at time of writing). The speed and rhythm he had going really left an impression, and changed my style. It caused a bunch of things to gel for me. Watching a pro do Unit Testing is a great thing. (Also check out John-Paul Boodhoo in the DNR TV Videos he and Carl made.)

Since then, I often write a method, and maybe the first thing I want to do is write a guard clause. I write a test to feed in the base value, then write the guard clause. I might have a series of bad scenarios, then finally, the normal flow, which is often my last unit test before deciding I am done (for now.)

So I end up with unit tests that look like this (maybe with ExpectedExceptions, but usually just verifying a return value or something):

[Test,ExpectedException] public void EdgeCase() {}

[Test,ExpectedException] public void EdgeCase() {}

[Test,ExpectedException] public void EdgeCase() {}

[Test] public void MainFlow() {}

Everything is very explicit, with a "statement" for every part of the contract the method implements...kind of like a verbose legal contract.

So the first thing I notice writing tests like this in mbunit is that these collapse down to:

[RowTest, ExpectedException, Row, Row, Row, ...] public void EdgeCase() {}

[RowTest, Row, Row, Row, ...] public void MainFlow() {}

It's about 30% faster for me to write the RowTest version, even though I have a CodeRush template for simple unit tests and expected exception tests. RowTests are just plain more concise. More like notesy instructions to your coding buddy than formal legal contract.

The only thing I don't like about this is the way the code looks formatted. Readability is very big for me, and I like collapsing my tests up so they look this this:

[Test] public void SomeMethod_ShouldDoSomething() ...

[Test] public void SomeMethod_ShouldDoSomethingElse()...

[Test] public void SomeMethod_ShouldntDoSomethingBad()...

All the Rows end up pushing the method name out of the right margin, or force me to put in line breaks.

But that's just being nitpicky. I'm sure I'll figure something out.

Currently listening to Party For Your Right To Fight by Public Enemy from the album It Takes A Nation Of Millions To Hold Us Back

Roy Osherove (ISerializable) has a post discussing using MbUnit vs. NUnit. I was a big fan of MbUnit when I first discovered it. The problem I found, almost exactly as Roy describes it, is that nobody has heard of it, and just getting people to use NUnit is pain enough.

The thing that NUnit really has going for it is that it's so simple. You can literally explain how to use it in a minute. It's only after a few days of serious use that people start to ask questions that MbUnit answers.

If I was fully in control of a project I was on, or I was developing something by myself, I would use MbUnit. In all other situations, I try to avoid the "yet-another-weird-library-from-Chris" syndrome and stick with NUnit.

This makes me sad, because I believe that you should always use the best tool for the job, not settle on something just because you are too lazy to explain something you know is superior. 

Why are people so afraid of new tools?

How can you improve if you never try anything better?

Link to ISerializable - Roy Osherove's Blog : MbUnit vs. NUnit Vs. Team System Unit Testing - Choosing a unit test framework

Currently listening to Tracy by Mogwai from the album Young Team

It's been a while since I tried the latest RC/Beta build of TestDriven.NET, one of my favorite tools ever. I read Jamie Cansdale's (the developer) weblog, so am aware of the features constantly being added, but I was still using a much older version, and hadn't tried any of the new features.

Since TestDriven.NET has become an essential part of my development process, I decided I really needed to buy a real license. A check I was waiting on just cleared, so I bought it this evening.

It's extremely pleasant to see a bunch of features now in TestDriven.NET that I had always wanted:

• Repeat Last Test Run: == New Hot Key. How many times I have jumped down into some code for a failing test, then spent 20 seconds finding my way back from some change (why hadn't I dropped a code rush marker! doh!). This rocks and is a tremendous time saver.
• Unhandled Exceptions don't end the test process. This is the thing that reminded me to download it. I have some code that calls an unmanaged C++ library that one of my tests was crashing, sort of on purpose. Now I will get better information about this than the usual "existing connection broken."
• 64-bit Support: I didn't know this was missing until after installing 64-bit XP last year, just before a presentation (doh!)
• Can Use MSBee Targets to Build: Awesome. We still have several projects that we need to compile in 1.1 and we often find one or two tests in a big suite that fail mysteriously on the build server, after being build for 1.1. Not to mention it will be easier to verify the 1.1 build works without running MSBuild manually.

Awesome stuff!

Link to TestDriven.NET > Home

I've got to admit that the first time I hear about Extension Methods, there was something I didn't like about them. It was my linguistics class earlier this year, that actually helped me get over my initial trepidation - extension methods are a grammar change, not a change to semantics. Encapsulation and information hiding still works. With extension methods, you are just changing the word order form Method-Argument to Argument-Method.

Anyway, Scott Bellware posted on using extension methods to implement a specification language:

With RSpec in Ruby, I could code a specification (test) that looks like:

user.username.should_equal "username"

With NSpec in C# 2.0, I would code the same specification as:

Specify.That(user.Username).ShouldEqual("username")

With C# 3.0, using an extension method created specifically for NSpec, the same specification could be coded as:

user.Username.ShouldEqual("username")

Just to be clear, the Username property in C# is an instance of System.String.  The extension method doesn't really add new behaviors to the String class directly, but it does allow for an arguably better DSL for NSpec by making use of this new feature in the C# 3.0 compiler.

This is one of the cooler examples of how one could use extension methods to make code more readable. I really dig this and can't wait for C# 3.0.

Source: A Ruby-like Mixin for NSpec Using C# 3.0 Extension Methods
Originally published on Wed, 13 Sep 2006 05:29:00 GMT