Ok, Roy wins, I'm starting to see the advantage of mocks with expectations

Last week I posted an entry about the simple, hand built, mocks that I use for testing in C++. Whilst I stand by my previous assertion that writing the mocks yourself gives you an further insight into your design from a client perspective I think the discussion that I had with Roy Osherove after last week’s posting has made me appreciate the power of having mocks that support the programmatic specification of ’expected’ behaviour.

Recap: To make my mocks easy and quick to write and allow my tests to confirm that the correct parameters have been passed to the function calls in question the mocks add lines to a simple text log as functions are called on them. At various points through the test I can verify that the log that the mock has created contains the correct data and therefore that the interaction between the object under test and the mock is as expected. Originally this checking was a simple case of passing in a string of expected interactions but for more complex interactions with more complex mocks this became cumbersome so I started to store the expected results on disk and have the test check the mock’s log from the disk file. If the check failed then the mock would write out its version of events and this made for an easy way to update the tests when code changes caused the interactions to change.

I’ve spent the week investigating and then working on a way of specifying my expectations in the code, like NMock2. I’m far from having a complete solution but I’m near enough to know that once I do my tests will be less brittle and my mocks will be more flexible.

The problem with logging information comes when you have some parameters that you dont want to know about sometimes but you do at other times; I find this usually happens when I move up a layer in testing. With text logs it’s hard to vary the detail level between tests and as soon as you need to do so the logging starts to get more complicated. If you’re specifying your expectations then you need to deal with all of that complexity to get the system to work at all ;) which means that once it’s done you don’t need to revisit that complexity for each mock object… Rather than specifying what each mock logs you specify what you’re interested in and allow each mock to ’log’ everything.

Say we have a mock with a method called “CreateEvent” and our class under test will call this method once. We can specify this with some code like this:

   myMock.Expect(_T("CreateEvent"));

This says that we’re just interested in the function call occurring. If we’re interested in all the parameters that are passed in to this call then we might use something like this instead:

   myMock.Expect(_T("CreateEvent"))
      .AddArgument(_T("lpEventAttributes"), CNullPointer())
      .AddArgument(_T("bManualReset"), CFunctionCallData(FALSE))
      .AddArgument(_T("bInitialState"), CFunctionCallData(FALSE))
      .AddArgument(_T("lpName"), CNullPointer());

If the call should have a valid name supplied then the code might look like this:

   myMock.Expect(_T("CreateEvent"))
      .AddArgument(_T("lpEventAttributes"), CNullPointer())
      .AddArgument(_T("bManualReset"), CFunctionCallData(FALSE))
      .AddArgument(_T("bInitialState"), CFunctionCallData(FALSE))
      .AddArgument(_T("lpName"), CFunctionCallData(_T("MyName")));

Each of these expectations is more specific and each could be expressed using a log but we’d have to have two different options for the mock object, one that says log your parameters and one that says don’t. If we find that we don’t control how the name parameter is set, the name is generated automatically for us, then we may wish to be specify less restrictive with our constraints.

   myMock.Expect(_T("CreateEvent"))
      .AddArgument(_T("lpEventAttributes"), CNullPointer())
      .AddArgument(_T("bManualReset"), CFunctionCallData(FALSE))
      .AddArgument(_T("bInitialState"), CFunctionCallData(FALSE));

which allows a null name or any name.

   myMock.Expect(_T("CreateEvent"))
      .AddArgument(_T("lpEventAttributes"), CNullPointer())
      .AddArgument(_T("bManualReset"), CFunctionCallData(FALSE))
      .AddArgument(_T("bInitialState"), CFunctionCallData(FALSE))
      .AddArgument(_T("lpName"), CNotNullPointer());

which allows any non null name…

When you need this level of flexibility and you’re using simple logging mocks the mocks become less and less simple and what’s more the complexity of the specification is duplicated; once in the mock and once in the test results. By using expectations we can remove the duplication. The mock always logs its parameters in the same way and only the expectations vary. Of course to get to this point I needed to put a little more work in up front; and it’s taken me some time to write a logging mechanism that that the mocks can use in this way, but now that it’s done it’s almost as easy for the mock to log its interactions as it was when using simple text logs.

So, thanks for making me think a bit more Roy, much appreciated. :)