Tangled testing?

| 2 Comments

Last night I started on my POP3 client code. I didn't know where to start; I wasn't really in the mood, so I wrote a test... That got the ball rolling. As usual the test first thing left me with a nicely decoupled class to write. Now I'm in a bit of a quandary about how to mock up the server that I need to test the client against...

The point of the CPOP3Client class is to provide a programmatic way to send POP3 commands and receive the results, parse them and present them in a way that's easy to manipulate programmatically. So we have a function called STAT() which makes a call to the server, parses the results and returns a StatResult which contains the number of messages on the server and the total size in octets. It will probably throw and exception if the call fails for any reason... All pretty straight forward stuff. My original plan was to make the client completely async, but I couldn't write a test for it, so I decided I didn't understand the problem enough and delayed that part of the work. A synchronous client will allow me to get the job done and I can go back to the async design when I understand what I want to do...

The client talks to the world via two interfaces. There's an output stream interface which allows it to write text and an input stream interface that allows it to read lines of text. At first I thought that the testing could involve me simply comparing whatever the client had written into the output stream with whatever I expected it to write, and priming the input stream with a response for it to parse and then checking that it behaves in the correct way. I expect I'll do this for some of the tests, but it means building more testing infrastructure.

What I'm considering doing is using the testing infrastructure that I already have for the server side tests and hooking that, and the real production server code up to the client. In effect I'd be testing the client by running it against the server with some simple input and output shims to link the two together. I can use all of the mock mailbox and message store stuff that I already have and look at the results from the client rather than the server. The server itself will act as the validator of the client's commands and I just have to write the code to validate the parsed responses.

I like this idea, but it worries me a bit. If there's a bug in the server then I'm writing a client that works because of that bug... This adds an element of risk, but I think it's one I'm prepared to live with given the kick start this will give me...

2 Comments


My original plan was to make the client completely async, but I couldn't write a test for it, so I decided I didn't understand the problem enough and delayed that part of the work. A synchronous client will allow me to get the job done and I can go back to the async design when I understand what I want to do...


And this is why I hate TDD, testing is a great thing. But testing too early is bad, and you are obviuosly doing that. First you need to know what your code has to do in full. For instance even if you wanted to have both sync. and async. APIs (I personally abhore sync. APIs due to non-scalability) the obvious implementation is to have something like...



API_sync_foo(X)
{
API_async_foo()
API_async_wait()
}


TDD's solution to this seems to be to completely rewrite your code when you get to the point of having to test the async. behaviour.


However this encourages the mindset of the "smallest addition". Where you do things like create things sync. knowing it's all not going to be fast enough in real life, so the "smallest addition" is to create a bunch of threads to deal with each connection in a syncronous way. Then you find out you want to read from the connection while writing, so then the "smallest addition" is to keep the group of connection threads and add some reader connection threads which the "main" connection threads just grab data from when they need it.

Of course by this point all the threads make it almost impossible to test the locking, and you could have had all the same functionality in half the code using normal async. io from the begining.

James, thanks for the comment. My response grew a little long, so I posted it as a new entry: http://www.lenholgate.com/archives/000195.html

Leave a comment