<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
    <title>Rambling Comments</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/" />
    <link rel="self" type="application/atom+xml" href="http://www.lenholgate.com/blog/atom.xml" />
    <id>tag:www.lenholgate.com,2010-12-10:/blog//12</id>
    <updated>2010-12-24T04:42:37Z</updated>
    
    <generator uri="http://www.sixapart.com/movabletype/">Movable Type Pro 5.12</generator>

<entry>
    <title>A single responsibility, please</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2005/07/a-single-responsibility-please.html" />
    <id>tag:www.socketframework.com,2005:/blog//12.518</id>

    <published>2005-07-20T17:28:32Z</published>
    <updated>2010-12-24T04:42:37Z</updated>

    <summary>Having got the CMessageProcessor under test in the last posting. I added a few easy tests for the object and then I came to another hard to add test. The reason that it was hard to add was that the...</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Socket Servers" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[Having got the <code>CMessageProcessor</code> under test in the <a href="http://www.lenholgate.com/archives/000433.html">last posting</a>. I added a few easy tests for the object and then I came to another hard to add test. The reason that it was hard to add was that the object is doing a little too much.]]>
        <![CDATA[<p>The easy to add tests were for the object's <code>Initialise()</code> and <code>Shutdown()</code> functions; though whilst adding the tests I made a note to see if I could do away with the functions in the near future. I personally abhor objects with two-stage construction (ctor and then init method), but I can understand why I wrote it this way as the object does things that have thread affinity and the object itself is created before the thread that it has affinity with is... </p>

<p>The complicated test was for the <code>Process()</code> function. This function is the '<code>main()</code>' of the business logic processing of the server. It looked like this:</p>
<pre class="brush: cpp gutter: false">void CMessageProcessor::ProcessMessage(
   CAsyncSocket *pSocket,
   IBuffer *pBuffer)
{
   // lots of business logic dealing with message processing omitted...<p></p>

   if (pBuffer)
   {
      pSocket-&gt;Write(pBuffer); 
   }
   else
   {
      pSocket-&gt;Shutdown();
   }
}</pre>
<p>The function gets given a socket that represents the connection that it's processing a message for and a buffer which contains a complete, distinct, ISO8583 message. It does whatever it needs to do and, if it needs to, creates a response message in the same buffer and then sends it out via the socket. In very rare situations, such as being given a message that it doesn't understand, it simply closes down the socket connection.</p>

<p>Unfortunately the <code>ProcessMessage()</code> function is violating the <a href="http://butunclebob.com/ArticleS.UncleBob.PrinciplesOfObjectOrientedDesign">Single Responsibility Principle</a> (see also <a href="http://www.amazon.co.uk/exec/obidos/redirect?path=ASIN/0135974445&amp;link_code=as2&amp;camp=1634&amp;tag=ramcom-21&amp;creative=6738">Agile Software Development (2nd edition)</a><img src="http://www.assoc-amazon.co.uk/e/ir?t=ramcom-21&amp;l=as2&amp;o=2&amp;a=0135974445" width="1" height="1" border="0" alt="" style="border:none !important; margin:0px !important;" /> by Robert Martin). The <code>CMessageProcessor</code> class should, probably, just process messages and produce responses. By adding in the additional responsibility of sending those responses back to the client we introduce additional complexity into the class and make it harder to test or reuse. The worker thread class that calls methods on the message processor already deals mostly with sockets and buffers and so the work that the message processor does that relates to sockets is more its responsibility. By moving that code out to the calling class the <code>CMessageProcessor</code> class becomes simpler and more focused and the calling class becomes completely responsible for all things socket related.</p>

<p>The code is easily adjusted to something like this:<.p>
<pre class="brush: cpp gutter: false">
bool CMessageProcessor::ProcessMessage(
   IBuffer *pBuffer)
{
   // lots of business logic dealing with message processing omitted...<p></p>

   return ok;
}</pre>
<p>The calling code is then adjusted to send the response or shut down the connection depending on the result returned by the message processing function. Our message processor is now devoted to one thing, processing messages, rather than two, processing and sending messages. This has made it easier to test the <code>ProcessMessage()</code> function as we now don't have to pass a socket, or a mocked up socket, into the function. We simply pass in a message and we get a message back in the same buffer. Of course we could have simply changed the concrete socket to an instance of <code>IAsyncSocket</code> and passed in a mock socket to use in testing, but doing so wouldn't have improved the design...</p>

<p>Our first test for this function now looks a bit like this:</p>
<pre class="brush: cpp gutter: false">void CMessageProcessorTest::TestProcessMessageX()
{
   CMockErrorMessageLog errorLog;

   CMockDatabaseConnectionFactory connectionFactory;

   CMessageProcessor processor(errorLog, connectionFactory);

   processor.Initialise();

   connectionFactory.CheckResult(_T("|CreateConnection|"));

   CMock0200Message message;

   THROW_ON_FAILURE(true == processor.ProcessMessage(&amp;message));

   processor.Shutdown();

   connectionFactory.CheckResult(_T("|DestroyConnection|"));

   errorLog.CheckNoResults();
}</pre>
<p>You can ignore the <code>CheckResult()</code> stuff, it's <a href="http://www.lenholgate.com/archives/000421.html">just simple C++ mock objects using text logging</a>; effectively declaring the expected function calls on the mocks.</p>

<p>The <code>CMock0200Message</code> needs to implement the <code>IBuffer</code> interface and allow me to build ISO8583 0200 messages in ways that will allow me to test the message processor. The problem is, <code>IBuffer</code> is a big interface and we only actually use a small number of the functions inside our message processor. </p>

<p><a href="http://www.lenholgate.com/archives/000386.html"><code>IBuffer</code></a> came about whilst some earlier refactoring of the socket server framework. Unfortunately when the interface was originally created it was created to allow for multiple different types of buffer to be used interchangeably; which it does. The thing is, it too is doing too much; the interface has two distinct uses. The two responsibilities of this class are 1) memory buffer management and 2) socket transmission bookkeeping data management.  The split between the multiple responsibilities can be clearly seen when you look at a client of the interface that only uses one half of the interface. Sure the interface has many more clients than the <code>CMessageProcessor</code> class but looking at them shows that they too either use one side or the other, with a few using mostly one side and a method or two from the other... </p>

<p>Perhaps it's because I have my <a href="http://staff.cs.utu.fi/kurssit/Programming-III/SRP(4).pdf">SRP</a> goggles on, but it looks to me that the interface should be split into two; one to manage the memory buffer and one to manage the socket transmission bookkeeping stuff. For now, for ease of performing this transformation I'll simply pull all of the memory buffer stuff out into a new interface and derive the old one from it. <code>ProcessMessage()</code> can now take an instance of the new interface and <code>CMock0200Message</code> becomes much easier to implement.</p>

<p>After implementing a bare bones version of the mock 0200 message class and getting the above test to fail I realise that it's time to change my focus a little. The <code>ProcessMessage()</code> test fails because the existing business logic that remains from the server shell that we started with knows how to spot an 0200 message and deblock it into a <code>CMessage0200</code> object for further processing. Looks like I should continue to develop the mock message whilst using it to test the real message; given our ISO8583 library it's trivial to put together message classes from specs and that was one of the first thing I did when I started to look at the spec, now I can write some tests for that work and by doing so validate the mock message that I'll use to test the message processor...</p>

<p>Sticking to the Single Responsibility Principle means that code is more cohesive; it only does <i>one</i> thing! This makes understanding and testing of the code easier because you only have one concept to understand or test at a time.</p>]]>
    </content>
</entry>

<entry>
    <title>More Socket Server Refactoring</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2004/10/more-socket-server-refactoring.html" />
    <id>tag:www.socketframework.com,2004:/blog//12.473</id>

    <published>2004-10-14T22:10:05Z</published>
    <updated>2010-12-24T14:05:50Z</updated>

    <summary>I&apos;m currently working on a simple auction server for a client. You can think of it as a specialised chat server, of sorts. One of the things it must do is broadcast messages from one user to a set of users. This is relatively easy to implement in a crude way in our socket server framework but it&apos;s not nice. Time to refactor towards niceness...</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Socket Servers" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[I'm currently working on a simple <a href="http://www.lenholgate.com/archives/000385.html">auction server</a> for a client using <a href="http://www.serverframework.com/">The&nbsp;Server&nbsp;Framework</a>. You can think of it as a specialised chat server, of sorts. One of the things it must do is broadcast messages from one user to a set of users. This is relatively easy to implement in a crude way in <a href="http://www.serverframework.com/">The&nbsp;Server&nbsp;Framework</a> but it's not nice. Time to refactor towards niceness...]]>
        <![CDATA[<p>The lack of niceness in the quick and dirty implementation of broadcast messages is the fact that the message must be duplicated for each client that will receive the broadcast. This sucks for many reasons; it's not especially elegant, it's not especially necessary, it involves doing work that needn't be done and it hogs memory. Unfortunately our last bout of <a href="http://www.lenholgate.com/archives/000237.html">server framework refactoring</a> stopped short of where we needed to be to be able to fix these issues.</p>

<p>The Server Framework uses fixed sized buffers that are managed by a buffer allocator (which can pool them for later use and pre-allocate a set number of buffers, etc). The buffer consists of a blob of memory that's used as the actual data buffer plus an <code>OVERLAPPED</code> structure for using the buffer in overlapped IO operations and a <code>WSABUF</code> structure for using the buffer in socket operations... When we wish to broadcast a buffer we want to reuse the memory blob and, possibly, the <code>WSABUF</code> but we need a unique <code>OVERLAPPED</code> structure so that each overlapped write operation has its own workspace... The framework could allocate its own <code>OVERLAPPED</code> and <code>WSABUF</code> structures and pool them, etc, but during the original design phase I decided to limit the number of memory allocations required during message processing by incorporating the buffer and all of its supporting structures in a single allocation - seemed like a good idea at the time and it's stood the test of time pretty well; until now.</p>

<p>What we'd really like to be able to do is attach multiple <code>OVERLAPPED</code> (and just to be on the safe side, <code>WSABUF</code>) structures to a single blob of memory. This would allow us to receive a message in a buffer and then transmit that buffer to multiple clients without needing to copy the message from the input buffer into each of the output buffers. </p>

<p>As usual the reason the existing design isn't quite flexible enough is because of the use of concrete classes rather than interfaces. The server deals in terms of <code>CBuffer</code> objects rather than in terms of an <code>IBuffer</code> interface. So, this morning, I slipped in an interface to decouple the users of the buffer class from its current implementation. That was reasonably painless, so we then added a 'read only buffer handle' class that implemented <code>IBuffer</code> and provided its own <code>OVERLAPPED</code> and <code>WSABUF</code> structures but references the original data in the original <code>IBuffer</code> implementation. This worked nicely and greatly reduced the memory usage of the server and improved the performance of the broadcasting. </p>

<p>The next step is to generalise the <code>IAllocateBuffers</code> interface so that we can share the pooling that we use for the real buffers. Once that's done it would be good to completely decouple the book-keeping parts of the buffer from the data parts, then, perhaps we can support buffers that consist of multiple memory buffers and that present arrays of <code>WSABUF</code>s to the socket layer...</p>]]>
    </content>
</entry>

<entry>
    <title>They&apos;re learning</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2004/09/theyre-learning.html" />
    <id>tag:www.socketframework.com,2004:/blog//12.458</id>

    <published>2004-09-09T22:02:16Z</published>
    <updated>2010-12-24T04:07:47Z</updated>

    <summary>I&apos;m back with the guys on the refactoring project for a couple of days. I got to my desk, updated my CVS tree and started to check my email. The first mail was from the boss man of the team; &quot;I fixed a bug in the FX code earlier in the week, we need to write a test for it so it doesn&apos;t happen again&quot;, followed by &quot;by the way, this time all the tests still run, I checked them all myself on Monday&quot;. This is considerably better than last time I visited them.</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[I'm back with the guys on the <a href="http://www.lenholgate.com/archives/000220.html">refactoring project</a> for a couple of days. I got to my desk, updated my CVS tree and started to check my email. The first mail was from the boss man of the team; "I fixed a bug in the FX code earlier in the week, we need to write a test for it so it doesn't happen again", followed by "by the way, this time all the tests still run, I checked them all myself on Monday". This is considerably better than <a href="http://www.lenholgate.com/archives/000314.html">last time</a> I visited them.]]>
        
    </content>
</entry>

<entry>
    <title>Reducing what the code could do to just what the code should do</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2004/08/reducing-what-the-code-could-do-to-just-what-the-code-should-do.html" />
    <id>tag:www.socketframework.com,2004:/blog//12.437</id>

    <published>2004-08-03T06:26:38Z</published>
    <updated>2010-12-23T17:06:23Z</updated>

    <summary>I&apos;ve got one of those &apos;please dig us out of this hole before we ship on Friday&apos; gigs this week. I&apos;m back with the refactoring project for a few days and, well, when the cat&apos;s away...

So, we&apos;re 80% done but none of it seems to work, no tests, and the design seems a tad overly complex and it just must go live by Friday... Our task is simple, just clean things up, make it work right and finish the other 80%.

The code is the usual mess of highly coupled, overly complex, unrestricted, unplanned, half understood, thought droppings; at least there&apos;s no XML...</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[<p>I've got one of those 'please dig us out of this hole before we ship on Friday' gigs this week. I'm back with the <a href="http://www.lenholgate.com/archives/000314.html">refactoring project</a> for a few days and, well, when the cat's away...</p>

<p>So, we're 80% done but none of it seems to work, no tests, and the design seems a tad overly complex and it just <i>must</i> go live by Friday... Our task is simple, just clean things up, make it work right and finish the other 80%.</p>

The code is the usual mess of highly coupled, overly complex, unrestricted, unplanned, half understood, thought droppings; at least there's no XML...]]>
        <![CDATA[<p>It seems that whenever I find myself in this situation the solution always ends up as being one where I systematically reduce what the code could do until it's doing just what it should do. Step one is usually removing unrequired optionality; replacing pointers that can't be null with references and removing any, now redundant, checking code. As soon as see a function that takes a pointer and immediately checks it for null you know you can remove the 1 or 0 optionality of a pointer and replace it with the certainty of a reference. Like retrofitting <code>const</code> correctness; once you do it in one place, you can chase back up the call stack until you find the point where the value really <i>is</i> optional. Things like this are simple to do but they vitally important in keeping code under control and understandable. By reducing the amount of things you have to question when you read a piece of code you can make the code easier to understand; no need to worry if this pointer can be null if you're given a reference that can't ever be.</p>

<p>I personally find it strange how some people write code that's loose and floppy by default; most things are public, most things are pointers but can never be null and are only sometimes checked, most things aren't correctly <code>const</code>, too much inheritance, lots of half thought out, 'what if it does this' functionality that's never used, completed or tested... Masses of undecipherable crap that can usually be boiled down to around a third of its original surface area by applying simple transformations that remove all the 'what ifs' that come from a lack of understanding. If you dont understand what the code should do, don't write code that might do what you might need, think it through and write the right code.</p>

<p>I much prefer reading code that is explicit about what it does. No ambiguity. No 'this is for when we do X in three versions time' stuff. This 'vapour design'  is usually just a hand waving that's unlikely to work and is just there so that someone can tick a box somewhere and other people can get really surprised when implementing X in three versions time takes far longer than everyone says it should... If an object can't be modified then it's <code>const</code>; if a pointer can never be null, then it's a reference; class data is private as are functions that a class uses to help it do its job but that are never called by clients... I don't want to look at code and wonder, I want to look at code and know.</p>

<p>So, once we've dealt with the low hanging fruit we can start on fixing the 'design'. Some people love inheritance so much that they just can't get enough; I used <a href="http://www.doxygen.org">doxygen</a> on the code to get it to draw some class and collaboration diagrams for me so I could start clicking through the mess in search of understanding. Lots of long thin class hierarchies that distribute functionality at random amongst the classes. It's interesting, in a way, it's like each day they'd come in and write some code to solve some of the problem but each day they'd derive from yesterday's work... </p>

<p>Not surprisingly there's lots of duplication. What is it with programmers and duplication? As usual, not all of it is cut and paste, that's easy to spot and easier to deal with, some of it is 'I don't remember how I implemented this last time, I don't even remember I have implemented this before, let's try it this way'; this is harder to spot, you need to change code until both areas look similar and only then do you get a chance to see the duplication. What's the deal with duplication and why do coders do it?</p>

<p>Luckily other transformations can help you spot the non obvious duplication. For example; if all loops look different because someone's working through 'my first book of looping constructs' and trying each one no more than once, you can start to remove noise by refactoring loops into a standard form. There really aren't that many ways you need to loop and if all your loops that do the same thing look the same you can spot the duplication and pull the code into one place. If you need to do several things before you can do what you actually want to do; such as access data in the 'one true data structure', calculate indices and then call one of the many methods of doing the same thing you can remove complexity by always doing these things in the same way and in the same order... </p>

<p>Code with no, or many, coding standards always benefits from having a standard applied, pick any one you like, it doesn't matter - no, really, it doesn't matter. Step one is to make code that does the same thing look the same, step two can be making it look like your version of 'nice'. If code that does the same thing looks the same, same block structure, same bracket rules, same names, etc, it's easy to spot duplication and once you spot it you can remove it.</p>

<p>We're one day in, we have 3 more to go, we're making some headway, but at present it's just low hanging fruit, I wonder if we'll make it...</p>]]>
    </content>
</entry>

<entry>
    <title>Back with the refactoring project</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2004/05/back-with-the-refactoring-project.html" />
    <id>tag:www.socketframework.com,2004:/blog//12.406</id>

    <published>2004-05-25T12:41:12Z</published>
    <updated>2010-12-23T16:48:50Z</updated>

    <summary>I spent a little time with the guys on the refactoring project last week. Of course, as is the way, pressure from the business for more functionality has reduced the amount of clean up work that they&apos;ve been able to do. The good news is that the builds are still repeatable and most of the tests pass. The bad news is only most of the tests pass.</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[I spent a little time with the guys on the <a href="http://www.lenholgate.com/archives/000220.html">refactoring project</a> last week. Of course, as is the way, pressure from the business for more functionality has reduced the amount of clean up work that they've been able to do. The good news is that the builds are still <a href="http://www.lenholgate.com/archives/000041.html">repeatable</a> and most of the tests pass. The bad news is only most of the tests pass.]]>
        <![CDATA[<p>The first thing I did after checking out the latest source from CVS was build all the test harnesses. I was pleasantly surprised that most of them built and ran without error. I was especially pleased (and relieved) that all the FX tests still worked. One new test harness had been written and a few tests. The tests that failed were failing due to a recent code shuffle. Unfortunately the shuffle was done for what at first appeared to be good reasons but later turned out to be merely a case of an inexperienced developer wanting to brush some complexity under the carpet.</p>

<p>The failing test failed to build because the interface to the object had been changed and the test hadn't been updated to accommodate the change. The interface change removed an interface that was being explicitly passed in to the constructor and then passed to a child object and instead had the child object run off and grab the interface from a process-wide static singleton object that was rapidly accumulating complicated and unrelated functionality.</p>

<p>The programmer responsible explained how this made things "better" because the relationships between the objects were less complicated. I pointed out that the complexity and coupling was still there it's just that now it was less obvious. The implicit coupling via the singleton meant that the object was now impossible to test... I explained how the explicit parameterization allowed us to test the object using a mock service provider and that now we'd have to delete this test because it wasn't possible to replace the singleton. At that the programmer agreed that the old way was better, he liked tests, and he just wished he had enough time to write some...</p>

<p>Whilst I was with them the project manager mentioned that there was a rather urgent new requirement for the FX code. It was yet another "special case"; one that had been missed when we'd done our initial work. We added the feature, test first, in a couple of hours; which was pretty cool...</p>

<p>First we adjusted the mocks so that we could set up the environment that was required. It was a little unusual; for some currencies, some traders only contributed the over night rate and didn't bother with the tomorrow next rate. The software was correctly dealing with this situation and interpolating the tomorrow next rate but it was then adjusting the overnight rate and it shouldn't have been. We adjusted the <a href="http://www.lenholgate.com/archives/000147.html">mock live data sources</a> so that we could set up a currency with no TN point. We ran the test and got the expected, wrong, results. We then debugged into the code and worked out why it was wrong and fixed it; when displaying the ON spread, don't subtract TN from ON if TN was interpolated... Ran the test again and we got the results that the traders required.</p>

<p>The interesting thing, for me was how the tests acted as compiled documentation. It's been 6 months since I've looked at the code and yet it was easy to work out how to make the changes required because the tests showed us how the objects worked and the 'documentation' that they provided was provably correct - the tests ran and passed. Working out where to make the change was easy because the first step was obvious; take the test for ON and make a test for ON without TN... Once that was done we could stick a few break points in and step through the calculations as the mocked up live data feed changed. I'm glad nobody had added any cancerous singletons into this particular area of the code.</p>]]>
    </content>
</entry>

<entry>
    <title>Admitting that your baby&apos;s ugly</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2004/01/admitting-that-your-babys-ugly.html" />
    <id>tag:www.socketframework.com,2004:/blog//12.331</id>

    <published>2004-01-27T11:12:54Z</published>
    <updated>2010-12-20T15:25:15Z</updated>

    <summary>I have a couple of days to myself. We&apos;ve just shipped some code to a client a couple of days ahead of schedule and we&apos;re waiting to recieve a purchase order from another client so I find myself without any client work to do. I&apos;ve decided to try and refactor the socket server code that we&apos;re using a lot right now. Whilst working on the code that we&apos;ve just shipped I realised that the new code I was writing was much easier to test than the socket server library that formed a major part of the project, so now that I have some time I&apos;m going to try and rectify that. The problem is, it means facing up to some unfortunate facts...</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Socket Servers" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[<p>I have a couple of days to myself. We've just shipped some code to a client a couple of days ahead of schedule and we're waiting to recieve a purchase order from another client so I find myself without any client work to do. I've decided to try and refactor <a href="http://www.serverframework.com/">The&nbsp;Server&nbsp;Framework</a> code that we're using a lot right now. Whilst working on the code that we've just shipped I realised that the new code I was writing was much easier to test than the <a href="http://www.lenholgate.com/cgi-bin/mt/mt-search.cgi?IncludeBlogs=12&Limit=20&search=socket+server">socket server library</a> that formed a major part of the project, so now that I have some time I'm going to try and rectify that. The problem is, it means facing up to some unfortunate facts...</p>]]>
        <![CDATA[<p>The <a href="http://www.lenholgate.com/cgi-bin/mt/mt-search.cgi?IncludeBlogs=12&Limit=20&amp;search=socket+server">socket server code</a> has had a much longer and more reusable life than we initially expected. We first wrote the code as part of a project for a client back in early 2002. We decided that the code would be more useful to us if it was a little more reusable so we cleaned it up and wrote about it on <a href="http://www.codeproject.com/script/articles/list_articles.asp?userid=133">CodeProject</a>. This led to further clients that wanted projects based on the codebase and, well, requirements changed and the thing grew. We added filtering, encryption and SSL support and produced UDP and AcceptEx versions of the code for various clients and wrote shims so that a client who originally bet the farm on the <a href="http://www.codeproject.com/internet/ndk.asp">NDK framework</a> could replace it for performance and stability reasons without changing any of their dependant code.</p>

<p>The result is code that's been pulled and pushed in all manner of directions that were never imagined during the original design stage. It still works, but it's more complex than it needs to be and it's hard to test...</p>

<p>Sometimes you get too attached to some code, because of the attachment you excuse problems that you would normally complain about and fix... It's time to take the socket server code apart and rebuild it. The code is poorly factored in places, the coupling is too tight and (I think I've said this before) it's too hard to test... </p>

<p>The good news is that I have lots of code that's dependant on this code and all of the servers have test harnesses that test the servers as a black box. So I am comfortable that I have a good test suite. The bad news is that I have lots of code that's dependant on this code...</p>]]>
    </content>
</entry>

<entry>
    <title>End of the refactoring project</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2003/12/end-of-the-refactoring-project.html" />
    <id>tag:www.socketframework.com,2003:/blog//12.316</id>

    <published>2003-12-20T11:42:14Z</published>
    <updated>2010-12-23T16:29:32Z</updated>

    <summary>My time working on the refactoring project has come to an end; at least for a while. Here&apos;s a little look back over what we achieved.</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        My time working on the refactoring project has come to an end; at least for a while. Here&apos;s a little look back over what we achieved.
        <![CDATA[<p>I've been working on the "<a href="http://www.lenholgate.com/cgi-bin/mt/mt-search.cgi?IncludeBlogs=12&Limit=20&amp;search=refactoring">Refactoring Project</a>" for around 9 months now. I've worked alongside the team responsible for the project several days a week for most weeks of that period. In that time we've tried to take a project that had become a <a href="http://www.lenholgate.com/archives/000128.html">big ball of mud</a>, whose original developer had left, and make it more maintainable and testable and correct. Whilst doing this we had to continue to add new functionality for the users regularly.</p>

<p>Things have gone well. At the start I produced a report on the key issues that were generating risk on the project and we've been working at reducing or eliminating that risk. </p>

<p>The first thing we did was introduce a repeatable build. Up until that point the team used CVS for revision control but didn't use tags for version control. Since that point we've done over <a href="http://www.lenholgate.com/archives/000040.html">42 releases</a> in 9 months. Each of those releases is <a href="http://www.lenholgate.com/archives/000041.html">repeatable</a> and we can still build all of the distinct releases if we need to. All releases to production are done via a script which is executed with the name of the version tag and this script pulls the correct source out of CVS and builds and packages the product for production deployment.</p>

<p>Once we had a controlled build environment so we knew what we were releasing we started to reduce the coupling in the code base. During the original development no thought had been given to coupling. Lots of classes were coupled to lots of other classes that they shouldn't really have cared about. Changing a single class often caused the whole project to rebuild as the header file dependencies were very coarsely grained. We've improved that. Classes keep more things to themselves; files only include what they really need to. Code complexity and coupling is down. Cohesion is up. <a href="http://www.lenholgate.com/archives/000214.html">Build times have dropped</a> from 13 minutes to 9 minutes.</p>

<p>The next major improvement was to separate the code base out into libraries of related functionality. The original design was a single OCX which contained hundreds of source files. We started a process or moving code into libraries not for reuse but just so that the physical organisation mirrored the logical organisation. The resulting libraries were more testable because we could simply build a test harness and link it to the library of code that needed to be tested.</p>

<p>During all of this we aggressively removed <a href="http://www.lenholgate.com/archives/000048.html">dead code</a> to simplify the codebase. There was lots of obviously dead code and lots of code that could never be executed because functions included unrequired flexibility that was never needed.</p>

<p>Once we had code in libraries we could start to test it. We were careful to introduce tests in a manner that provided the most "bang for the buck". We didn't aim for broad coverage, we simply aimed to introduce tests into code that we needed to fix or work on. The first test was in the <a href="http://www.lenholgate.com/archives/000059.html">data entry validation code</a>; this allowed us to improve the performance of an existing piece of code whilst validating that we hadn't changed the expected functionality. </p>

<p>Once we had a test and a test harness there was a point to doing a <a href="http://www.lenholgate.com/archives/000061.html">daily build</a>. This was the simplest possible daily build. I adjusted our release script to be able to build and run the test harnesses and simply ran the script on the head revision of the source code every day. The daily build ran on my machine and was manually instigated, far from ideal, but better than nothing.</p>

<p>During the time that it took to implement all of the above processes we also spent some time <a href="http://www.lenholgate.com/archives/000063.html">managing our user's expectations</a>. We wanted them to know that we <i>could</i> release new features quickly if they <i>really</i> required them quickly but that it was safer for them and more efficient for us to release them to a predictable schedule. This took some time to take hold because up until this point some users had been used to calling up, having someone make a fix and for that fix to just suddenly be available in production. No tagging, no official build, no testing... Eventually they trusted us to slow the perceived rate of new feature releases down in order to <a href="http://www.lenholgate.com/archives/000076.html">increase the actual rate</a> of new <a href="http://www.lenholgate.com/archives/000081.html">features released</a>...</p>

<p>At around this time my focus shifted from the <a href="http://www.lenholgate.com/archives/000105.html">entire application</a> to one particular area of the app; the <a href="http://www.lenholgate.com/archives/000130.html">FX trading</a> displays and rates calculation engine. This area had a distinct set of users and they needed a lot of changes made quickly. The plan was to switch all FX trading from two other vendor systems to the refactoring project. </p>

<p>The FX code was a complex mess. It was complex mainly because the original developer didn't appear to fully understand the business domain. I tend to stay pretty technically focused and only learn as much as I need to about the business to write the code I need to write; so for the work on this piece of code I went out and bought a <a href="http://www.lenholgate.com/archives/000140.html">book on FX trading</a>.</p>

<p>Once I understood what we were trying to do we <a href="http://www.lenholgate.com/archives/000142.html">teased the code apart</a> and into a library and put <a href="http://www.lenholgate.com/archives/000145.html">some tests</a> in place. The tests for the FX code required that we mock up the data provider so that we didn't have to try and debug code that was being driven by <a href="http://www.lenholgate.com/archives/000147.html">live market data</a>. Once that was done we could write <a href="http://www.lenholgate.com/archives/000150.html">all manner of tests</a> for all the complicated edge cases and drive them with our mock data provider. The user interface for the FX GUI <a href="http://www.lenholgate.com/archives/000152.html">proved harder to test</a>. With the tests in place at the inflection point, the point where the results of the changes we were making met the rest of the code, we decided to release the current code changes before we began refactoring to clear the way for adding new functionality. We went into UAT with the new code and found that our <a href="http://www.lenholgate.com/archives/000155.html">testing wasn't quite as good</a> as it <a href="http://www.lenholgate.com/archives/000161.html">could have been</a>.</p>

<p>Now that we had some tests to support our FX changes we <a href="http://www.lenholgate.com/archives/000191.html">refactored </a>and added new functionality and from that point on things went well. The tests gave us confidence and we added new functionality quickly whilst fixing bugs and generally improving the quality of the code. The users were very pleased with the speed that we could react to challenging <a href="http://www.lenholgate.com/archives/000217.html">new requests</a>.</p>

<p>And that's about it. There's a lot more to be done on the project but now is the time to leave the team to get on with it alone. The users are happy. The client is happy and we may rejoin the team to help them out some more some time next year.</p>]]>
    </content>
</entry>

<entry>
    <title>Bug hunt on the refactoring project</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2003/11/bug-hunt-on-the-refactoring-project.html" />
    <id>tag:www.socketframework.com,2003:/blog//12.310</id>

    <published>2003-11-21T08:21:27Z</published>
    <updated>2010-12-20T16:14:41Z</updated>

    <summary>The refactoring project rolls on. Mostly it&apos;s been more of the same so I haven&apos;t bothered boring my reader with the details. This week we had an interesting bug to fix. The bug had appeared in a much earlier version, way back in July, but it had only been reported by one user and we could never duplicate the problem. This week we managed to duplicate it, and then we needed to work out what it was and when it was added to the source...</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[<p>The <a href="http://www.lenholgate.com/archives/000179.html">refactoring project</a> rolls on. Mostly it's been more of the same so I haven't bothered boring my reader with the details. This week we had an interesting bug to fix. The bug had appeared in a much earlier version, way back in July, but it had only been reported by one user and we could never duplicate the problem. This week we managed to duplicate it, and then we needed to work out what it was and when it was added to the source...</p>]]>
        <![CDATA[<p>The bug was a hard one to fix, mainly because it was a hard one to understand. The refactoring project is an OCX and runs inside IE. This wouldn't have been my first choice for architecture but that's what we have to live with. The bug manifested itself as a complete lock up of the OCX when a user pressed the escape key. The OCX wasn't dead and no exceptions occurred, it just didn't get any more messages to process...</p>

<p>It never used to do that... But we couldn't work out when the problem was introduced. I looked at our release history in CVS and started a binary search of previous releases to try and locate the point where the problem started.</p>

<p>We have a 'one click' release procedure. So building the previous versions was just a case of <code>release Release_5000</code> and wait for the correct version of the source to be pulled from CVS and the build to complete. Whilst working through the numerous releases since I instigated the "<a href="http://www.lenholgate.com/archives/000052.html">it would be nice if we could know what we actually have in production</a>" policy I noticed that the older releases seemed to take an age to build...</p>

<p>I adjusted the build script so that it reported the amount of time taken by the build and discovered that the refactoring so far has shaved 4 minutes off of the build. Release 5000 takes 13 mins to build and release 8017 (just don't ask about the version number 'policy') takes 9 mins. Most excellent!</p>

<p>I eventually located the version where the bug was introduced and fixed it...</p>]]>
    </content>
</entry>

<entry>
    <title>Back to the refactoring project</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2003/10/back-to-the-refactoring-project.html" />
    <id>tag:www.socketframework.com,2003:/blog//12.278</id>

    <published>2003-10-01T19:24:25Z</published>
    <updated>2010-12-21T08:27:30Z</updated>

    <summary>I&apos;ve spent the last couple of days back with The Refactoring Project. They&apos;ve done well in my absence. They managed 3 releases; all correctly tagged and repeatable. They started some refactoring of their own and, at first glance, it looks like they&apos;ve taken on board lots of the suggestions I&apos;ve been making over the last months. They&apos;ve fixed a couple of new bugs in the FX code and whilst doing so found that a) the bugs were easy to locate, b) they were easy to fix with very localised changes, and c) the new code was much easier to work with! They even ran the tests after making the changes! All in all quite a reassurring result.

Now, of course, I have to move away from the relative safety of the newly refactored FX code (complete with tests) and back into the pile of code that has yet to be touched... Still, at least things are heading in the right direction.</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[<p>I've spent the last couple of days back with <a href="http://www.lenholgate.com/archives/cat_refactoring.html">The Refactoring Project</a>. They've done well in my absence. They managed 3 releases; all correctly tagged and repeatable. They started some refactoring of their own and, at first glance, it looks like they've taken on board lots of the suggestions I've been making over the last months. They've fixed a couple of new bugs in the <a href="http://www.lenholgate.com/cgi-bin/mt/mt-search.cgi?IncludeBlogs=12&Limit=20&amp;search=FX">FX code</a> and whilst doing so found that a) the bugs were easy to locate, b) they were easy to fix with very localised changes, and c) the new code was much easier to work with! They even ran the tests after making the changes! All in all quite a reassurring result.</p>

<p>Now, of course, I have to move away from the relative safety of the newly refactored FX code (complete with tests) and back into the pile of code that has yet to be touched... Still, at least things are heading in the right direction.</p>]]>
        
    </content>
</entry>

<entry>
    <title>You&apos;ll tick when I say so and not before!</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2003/08/youll-tick-when-i-say-so-and-not-before.html" />
    <id>tag:www.socketframework.com,2003:/blog//12.249</id>

    <published>2003-08-13T17:43:30Z</published>
    <updated>2010-12-21T09:38:32Z</updated>

    <summary>Today we wrote some complicated FX business logic tests. Things like making sure that the FX library can calculate a EURUSDCAD 1M rate - it can; or a USDCAD ON rate - it can&apos;t...</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Geek Speak" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Testing" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[<p>Today we wrote some complicated <a href="http://www.lenholgate.com/cgi-bin/mt/mt-search.cgi?IncludeBlogs=12&Limit=20&amp;search=FX">FX business logic tests</a>. Things like making sure that the FX library can calculate a EURUSDCAD 1M rate - it can; or a USDCAD ON rate - it can't and nobody had spotted the fact that it was out because it's way down in the 0.00001's of the rate.</p>]]>
        <![CDATA[<p>It has taken us around a week to get here. The initial code was a tightly coupled mess. The current code is still a mess; but it's less tightly coupled and it's detached from the GUI code. </p>

<p>The mock live data provider that we wrote last Friday is now fully operational. You can set it up to provide dummy data for a specific set of currencies, add holidays if you like, set "today" to whenever is convenient for the test in hand and then force particular data points to tick with whatever values you feel like specifying and completely under program control. If only we'd had this a month ago when we were trying to track down some of the more obvious errors in the FX rates code. </p>

<p>Tomorrow we instrument the existing 'correct' application so that we can capture some live data and the corresponding output that the FX engine produces. We'll use this data to write tests to prove that refactoring from the live version of the code to get to the testable version of the code hasn't changed anything. Once those tests pass we'll merge this set of changes into the main branch and release it to the users. Once that's done we'll start to verify that the results it produces aren't just in line with the last version but are actually correct as well!</p>

<p>To get to this point we've had to write a lot of support code. What's good is that much of the mock data provider code is likely to be usable when we start refactoring the real "live data" provider. The test harness code is written to be as simple as possible and it does around 75% of what the real data provider needs to do; hopefully we can just hoist it into the Reuters library and fill in the gaps. </p>

<p>The FX library is now in a state where the final bits of refactoring can be done. These are the scary parts of the code, the convoluted rates calculations that bear no resemblance to the examples in the FX books we've been using for reference. The good thing is that now it's not scary code to change. We have a full test harness for the library and we can experiment with reimplementing the calculation code when we're writing tests to test the existing code. I expect we'll see a lot more code moving from the test harnesses into the production code.</p>

<p>Let the refactoring commence!</p>]]>
    </content>
</entry>

<entry>
    <title>The first FX test</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2003/08/the-first-fx-test.html" />
    <id>tag:www.socketframework.com,2003:/blog//12.247</id>

    <published>2003-08-11T06:28:02Z</published>
    <updated>2010-12-21T09:41:57Z</updated>

    <summary>On Friday we got to the point where the FX buiness logic code was suitably decoupled from the display logic that we could write our first test for the business logic. In the words of Homer Simpson, &quot;Woo hoo!&quot;.</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Geek Speak" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Testing" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[<p>On Friday we got to the point where the <a href="http://www.lenholgate.com/cgi-bin/mt/mt-search.cgi?IncludeBlogs=12&Limit=20&amp;search=FX">FX buiness logic</a> code was suitably decoupled from the display logic that we could write our first test for the business logic. In the words of Homer Simpson, "Woo hoo!".</p>]]>
        <![CDATA[<p>Test one was a simple one; can we construct the FX 'live point' object. After adding a few interfaces and mocking up a service provider or two we could. </p>

<p>The object uses two-stage construction; not ideal, but the code doesn't use exceptions yet. The first stage creates the point itself and the second stage wires it up to the live data. The second stage can fail if we're asking for a point for which there is no live data, or for which the live data doesn't extend out far enough in the future. </p>

<p>Writing a test for the second stage of construction was more complex. Most of the parameters passed into the function should have actually gone into the constructor, they're constant for the life of the point and a point can only be constructed and made live once. Not the best of design, but once we have some tests we can look at dealing with that side of things. Whilst writing the second test we refactored the first test and the construction code to move these items back to the constructor. </p>

<p>Now the function needed a live data provider. We had one, but only a concrete class. We slipped in an interface, decoupled the FX code from the Reuters code and created a <a href="http://www.lenholgate.com/cgi-bin/mt/mt-search.cgi?IncludeBlogs=12&Limit=20&search=mock+object">mock</a> data provider. We then chased concrete Reuters classes through the FX code until they were all replaced by interfaces and we had a very basic set of mock live data classes.</p>

<p>The decoupling process was quite interesting. Step one was to locate a concrete class that we needed to mock up. Slip in an interface with no methods and have the things that needed the service accept the interface and the thing that provides the service derive from it. Recompile and the compiler tells you all the methods that need to be in the interface. Move all of these to the interface, and repeat for the next concrete class.</p>

<p>This seemed to be a particularly painless process and one that left us with interfaces that provided only the functionality that the clients required. It acted as a neat way of seperating the code that the Reuters library needed to manage the interactions between the objects with the code that the clients of the library used.</p>

<p>So now we had a set of simple, stubbed out, mock live data classes and an FX engine that used them. We could wire the FX code to the mock objects and watch the test fail as the FX code asked the data provider for live data that it knew nothing about.</p>

<p>We had a failing test, must be time to go home for the weekend.</p>]]>
    </content>
</entry>

<entry>
    <title>FX refactoring</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2003/08/fx-refactoring.html" />
    <id>tag:www.socketframework.com,2003:/blog//12.244</id>

    <published>2003-08-07T21:21:04Z</published>
    <updated>2010-12-21T09:49:51Z</updated>

    <summary>Bleugh! You are lost in a maze of crapy code, all alike (and much of it copy and pasted!). The last few days have been deep in the heart of darkness. Gently teasing the business logic and the display logic of the FX code apart so that we might one day be able to write tests for the business logic.</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Geek Speak" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[<p>Bleugh! You are lost in a maze of crapy code, all alike (and much of it copy and pasted!). The last few days have been deep in the <a href="http://www.lenholgate.com/archives/000079.html">heart of darkness</a>. Gently teasing the business logic and the display logic of the <a href="http://www.lenholgate.com/archives/000130.html">FX code</a> apart so that we might one day be able to write tests for the business logic.</p>]]>
        <![CDATA[<p>We're nearly there but it's been a long job. Much longer than any of the other refactorings that we've done. We're refactoring to make it testable and because it's currently broken (we don't do things correctly and all the edge cases are wrong!). It's scary stuff. There are so many things that make this code hard to test and it's so tangled together that you have to break things to get it apart.</p>

<p>I've ditched two pieces of functionality on the basis that they don't work particularly well at present and they'll be easier to add back in once the rest of the code is working. Ideally I'd have preferred to keep them functional all the way through, but, well, needs must.</p>

<p>We have some code that talks to the Reuters library. It subscribes to live FX data. A nice simple job. It does it poorly. Multiple tickers make up a single forward point: Everything's relative to spot, so we always need the spot ticker; some points are dates that are directly contributed - like the 1M point; whilst others are interpolated - like 1month and 1 day needing the 1M point and the 2M point. So a ForwardPoint needs 1-3 tickers. The actual point displayed in the grid might need multiple forward points - it may be a cross currency thing and so need a point for each currency pair; it may be a forward forward and so need a near point and a far point, etc. This was modelled in an "interesting" and sadly familiar way; lots of fixed sized vectors of stuff with no typedefs.</p>

<p>All of this rubbish ends up with a dialog that does some of the display work - I know it should probably do all of the display work, but that's not how they did things around these parts. So we have a dialog that's really a rates engine. Originally the dialog could run 'invisibly' but we nailed that one fairly sharpish and moved the real rates engine functionality into something that didn't need a UI. Even so the dialog has multiple "live points" which contain 1-n forward points each of which is 1-3 tickers. You can subscribe to tickers, so guess who does that - of course it's Mr Dialog... When the ticker ticked the dialog got told and then did some horrible hoop jumping to get from the ticker to the live point and thence on down to grab the data. </p>

<p>Let me hear you say "Bleugh!".</p>

<p>It's cleaner now. Not ideal, but better. The FX point and calc code is now in a library and it doesn't need a GUI. The dialog can subscribe to the live points, these subscribe to the tickers they need and they only poke the dialog once even if they have several points dependant on spot and spot ticks... </p>

<p>Still awake at the back?</p>

<p>Testing up until now is a case of running old and new side by side and making sure the numbers tick the same way. Not nice, but we're almost at a point where we can test the FX calcs in isolation; driven from data that we supply (captured from the existing 'correct' version of the app). Once we get there we can do some heavy duty (and safe) refactoring of the remaining mess. I expect the refactoring will result in not a single line of the original code remaining. We'll take nice small steps though.</p>

<p>The code is already an order or magnitute simpler than it was. So much complexity and twistyness just fell away when we rearranged the subscription model. Lots of wierdly complex ownership tangles fell to the floor when we wrapped up the collections into classes so that they weren't just <code>std::</code> containers spread thinly around other classes. Now they're wrapped up they can own their contents and an infinite amount of bollocks just vanishes.</p>

<p>I feel better now.</p>

<p>Some people shouldn't be allowed to write code.</p>]]>
    </content>
</entry>

<entry>
    <title>It&apos;s important who&apos;s driving</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2003/06/its-important-whos-driving.html" />
    <id>tag:www.socketframework.com,2003:/blog//12.209</id>

    <published>2003-06-24T06:29:06Z</published>
    <updated>2010-12-21T11:02:07Z</updated>

    <summary>I&apos;m a firm believer that software rots unless you&apos;re very careful; and like apples, once one piece starts to go bad the rest quickly follows. The Pragmatic Programmers talk about Software Entropy and The Broken Window Theory and, unfortunately, this week the refactoring project showed how true this is...</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Geek Speak" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[<p>I'm a firm believer that software rots unless you're very careful; and like apples, once one piece starts to go bad the rest quickly follows. <a href="http://www.pragmaticprogrammer.com/">The Pragmatic Programmers</a> talk about <a href="http://www.pragmaticprogrammer.com/ppbook/extracts/no_broken_windows.html">Software Entropy and The Broken Window Theory</a> and, unfortunately, this week the refactoring project showed how true this is...</p>]]>
        <![CDATA[<p>We've spent several weeks working on cleaning up this project, but there's a lot of code and it's all in a bit of a state. Things are getting better, we have <a href="http://www.lenholgate.com/archives/000059.html">tests</a>, we have a <a href="http://www.lenholgate.com/archives/000052.html">repeatable build</a> and the code is <a href="http://www.lenholgate.com/archives/000095.html">slowly being beaten</a> into something that will be maintainable. But the rot continues.</p>

<p>Now that we have our source control working properly we can safely let developers work on a branch and develop new functionality without worrying about them breaking other developer's work, or affecting regular releases or bug fixes, etc. Sounds good, but of course the time comes to integrate this new work back into the main branch... Since the other developers aren't that comfortable using branches and don't see the point I offered to help with the merge - mainly so that I could make sure that it was done properly... </p>

<p>The interesting thing about doing a merge is seeing what's been changed. What I found interesting was how the quality of the new code closely resembled the quality of the old code in the area where the changes had been made. Where changes were made in recently refactored code that was now in good shape the new code was nice and clean; where the changes were in parts of the code that were still a mess the changes were, well, in keeping with the existing style... </p>

<p>This wasn't just a misguided application of the old "make changes in the style of the original code" mantra; it was a quality thing. Where the surrounding code was nicely factored into small functions that did one thing and were named sensibly the changes had been applied in such a way that the new code was also nicely factored, changes were put into their own functions and called appropriately. Yet where a change had to be made to a 300 line function the change was just hacked into the middle of the function. </p>

<p>Where the code was good the developer would keep it good but where it was bad he wouldn't attempt to improve it. In fact, he'd accelerate the rot. This underlined to me just how important it is for a project's driver to be someone who cares. If the project had been started with a little more care and attention then these additional developers could have been added and the quality of work been maintained. Since things were allowed to go rotten very early on the additional developers just increased the rot.</p>

<p>We have a lot of work ahead of us...</p>]]>
    </content>
</entry>

<entry>
    <title>Developer buy in</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2003/06/developer-buy-in.html" />
    <id>tag:www.socketframework.com,2003:/blog//12.189</id>

    <published>2003-06-13T11:11:51Z</published>
    <updated>2010-12-21T12:00:38Z</updated>

    <summary>The refactoring project rolls on and the code gets better. This week saw a marked change in attitute from some of the developers on the team......</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Geek Speak" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        <![CDATA[<p>The refactoring project rolls on and the code gets better. This week saw a marked change in attitute from some of the developers on the team...</p>]]>
        <![CDATA[<p>I was originally brought into this project because the lead developer was leaving and other developers within the team were unhappy with taking over the project as they were scared of the state that the code was in. Management accepted the need for change and welcomed the incremental improvements that my original report on the project proposed. The other developers were unsure and, though they didn't want to take control of the project, they also didn't really want to change how they did things. This week that started to change...</p>

<p>People have got used to the idea of using a source control system properly. We've done several nicely controlled releases followed by patches to fix bugs. Whilst that's been going on we've progressed major refactorings on another branch and integrated the changes when we wanted to release. Things are much more controlled than a month ago and minor releases are no longer a time of stress and worry as we know exactly what has gone into each one. </p>

<p>One of the other developers is working on some new functionality on his own branch. He's come across a point where the existing code is in a very bad way and where he knows that just hacking in his changes will be bad for the project in the long term. He's under time pressure to complete his functionality. Up until last week I'd have expected him to just hack in the changes. The existing team have been in 'just get it done' mode for so long that even though they know they're building a rod for their own back, they still hack stuff in rather than think about the long term effect. </p>

<p>This week he called me and his manager into a meeting and told us straight that the area he's about to work on needs refactoring and that might take longer than just hacking in his fix. He pointed out that hacking in the change will make it very hard to continue to maintain this area of the code but that he doesn't think he can hit his deadline if he has to refactor the code. He suggested that I add this area of the code to our list of danger areas and that he hack in his changes to meet his deadline but look at refactoring after. We decided that since this was an important area of code we'd make it the highest priority for refactoring and that I'd look at refactoring it in parallel with him hacking in his changes.</p>

<p>Good result. The tide is turning.</p>

<p>Just after this the manager, who also codes, had a bug report to deal with and took a quick look at the code and decided it was an easy fix. He sent out an email to the users saying that they could have their bug fixed in a patch release that afternoon and then started to work on the bug. Unfortunately the bug was in <a href="http://www.lenholgate.com/archives/000079.html">The Heart of Darkness</a> - the most twisty of all the twisty code. His simple fix took a day and a half to get right due to unexpected side effects and a general lack of design in the area of code that needed fixing. He slipped his release by 2 days and we ended up pair programming to get it done. </p>

<p>I'd already flagged up the fact that simple changes in this area were likely to take far longer than expected due to the poor design but there's no better way to get this kind of point across than have the guy responsible for making the call try and make a simple change and fail ;) </p>

<p>The good news is that we've now added The Heart of Darkness to the refactoring list as a high priority item. The bad news is that it's still a very scary piece of code.</p>]]>
    </content>
</entry>

<entry>
    <title>We came. We saw. We did a little testing.</title>
    <link rel="alternate" type="text/html" href="http://www.lenholgate.com/blog/2003/05/we-came-we-saw-we-did-a-little-testing.html" />
    <id>tag:www.socketframework.com,2003:/blog//12.172</id>

    <published>2003-05-30T23:06:09Z</published>
    <updated>2010-12-21T12:32:55Z</updated>

    <summary>Another week another release. Well, almost. The plan was to release today. The plan ignored the fact that most of the team are at a wedding this weekend and nobody was around today and nobody&apos;s around on Monday......</summary>
    <author>
        <name>Len</name>
        
    </author>
    
        <category term="Geek Speak" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Refactoring" scheme="http://www.sixapart.com/ns/types#category" />
    
        <category term="Testing" scheme="http://www.sixapart.com/ns/types#category" />
    
    
    <content type="html" xml:lang="en-us" xml:base="http://www.lenholgate.com/blog/">
        Another week another release. Well, almost. The plan was to release today. The plan ignored the fact that most of the team are at a wedding this weekend and nobody was around today and nobody&apos;s around on Monday...
        <![CDATA[<p>The latest release included some new functionality and a lot of refactoring. Given the amount of code that had been 'touched' I recommended a longer test phase than we usually planned, i.e. more than 1 day. I was told that we didn't need more than a day of testing and that we probably wouldn't actually do more than a day even if we allowed more time. I explained that it wasn't the testing we needed extra time for; it was the fixing and retesting... No joy.</p>

<p>We came. We saw. We did a little testing. </p>

<p>We found a few bugs. Not that many, given the amount of code change and the lack of unit tests. Fixing, retagging, rebuilding from CVS and retesting took time. Rinse. Repeat.</p>

<p>Today we decided to defer the release until Tuesday. I expect we could have gone with the code we ended up with at close of play today. I advised another day's testing; we decided that was best... </p>

<p>I think an aborted release is a good thing. You don't need to do it all the time. But you need to be able to do it if you have to. The first is probably the hardest. Let people down. Realise that they don't kill you. Move on. The release on Tuesday will be solid, we have a couple more days to test the actual release process. A release today would have exposed the team to the risk of a failed release and all of the negative feelings that carries with it.</p>

<p>The whole process of deciding we're releasing. Creating a branch. Building and testing that branch. Fixing it. Continuing to work on new stuff on the main branch. Testing and fixing and retagging the release. Taking the final cut. Merging the ongoing work with the fixes. It was all a useful exercise. It's all stuff that's hard to organise. People make mistakes because they're not used to working on the release branch to fix bugs in the release. They're not used to branches. They don't really understand why we're using a source control system at all... So we do it. We screw up. We screwed up lots in the last couple of days. We fixed it. </p>

<p>I wrote tests to fix bugs. We have more test coverage. I didn't actually write the tests before fixing the bugs - we were too rushed ;) But I did write the tests before I merged the fixes into the main branch. That's good enough in my book. We have more tests. They test things that were broken. We're safer for next time...</p>

<p>We got ready for a release and then we didn't do it. That's good. When we do do it I expect it will pretty much work...</p>]]>
    </content>
</entry>

</feed>



