Today I used a variation on my job object based memory limiting test runner to investigate the original problem with commands spawned remotely using WinRS. Given that I had a test runner that could spawn a process and place it into a job object it was simple to adjust this so that the target process was instead broken out of any job that the test runner might be in; just add CREATE_BREAKAWAY_FROM_JOB to the process creation flags and hope... This seemed to work so I then looked into reporting on the job that the test runner found itself in... Some simple calls to IsProcessInJob() and QueryInformationJobObject() with null jobs showed that the WinRS processes were being run inside of a job and that the limits (on my particular box) were as follows:

Job report for process: 10676
Process IS in a job
Flags: 0x2b08 - 
   JOB_OBJECT_LIMIT_ACTIVE_PROCESS
   JOB_OBJECT_LIMIT_PROCESS_MEMORY
   JOB_OBJECT_LIMIT_JOB_MEMORY
   JOB_OBJECT_LIMIT_BREAKAWAY_OK
   JOB_OBJECT_LIMIT_KILL_ON_JOB_CLOSE

ActiveProcessLimit: 15
Affinity: 0x0
MaximumWorkingSetSize: 0
MinimumWorkingSetSize: 0
PerJobUserTimeLimit: 0
PerProcessUserTimeLimit: 0
PriorityClass: 32
SchedulingClass: 5
JobMemoryLimit: 157286400
ProcessMemoryLimit: 157286400

The problem limits being the job and process memory limits. Luckily the WinRS job has JOB_OBJECT_LIMIT_BREAKAWAY_OK set so my attempt to break the target process free of these limits works fine.

Of course, being part of boost I first have to work out exactly how much more of boost it will require me to depend on and then I have to work out how I can use it to replace my current std::map usage. It seems quite clever (no surprise there) and allows for a type to be included in multiple intrusive containers by allowing an object to have multiple links embedded in it and allowing you to specify the link to use for a particular container. It seems slightly over engineered for my needs (again, no surprise there) and it's a pity that it doesn't simply provide obvious drop in replacements for the equivalent STL containers; yes it's nice that there are two kinds of base tree structure (red black and AVL) so you can optimise for your specific usage patterns but it would be nicer if you could simply switch from std::map to boost::intrusive::map and add an appropriate data member to the class you want to store in it...

A quick Google led me to Pete Isensee's pages where he talks about STL allocators for games programming. He has a nice set of slides and some code and this has helped me get started.

I can't help thinking that allocators weren't really thought through especially well; the whole 'rebind' from allocator<int> to allocator<node> is, IMHO, just pointless 'aren't we clever with templates' crap ;) especially given the fact that at the end of the day all these things are doing is allocating memory of a given size...

The problem with tools that lag is that no matter how useful they are the lag and the lack of daily use affects the usefulness of the tool and that always gets evaluated when I have to pay more money for another year of 'support'...

When a tool isn't in the tool box that you use daily you tend to use it less often than you should... Yes, the reason for this rant is that I've just found a couple of leaks with DevPartner that lived a little longer than they should because I'm developing the code in VS2010 and DevPartner isn't there in the toolbar nagging at me to use it. And, yes I know that I'm the one at fault for not using the tool anyway even though it's easier to ignore right now...

This kind of thing usually isn't actually too much of a problem for me as using these kinds of tools is on my release check-list but as I get clients that only want VS2010 builds it gets more likely that the tool lag will bite.

The offending code was a wrapper around the operating system function GetTempPath(). I always tend to wrap these kinds of things so that I can convert errors to exceptions (I did a good job of that here didn't I) and so that I can convert types to the kind of types that I prefer to use. In this case I also dealt with the fact that you need to call the API without a buffer to determine how big a buffer it requires and then call it again with the correct sized buffer to obtain the path.

GetTempPath() is a simple function and the documentation is pretty clear on how it should be used. My original wrapper was fairly crap.

_tstring GetTempPath()
{
   const DWORD spaceRequired = ::GetTempPath(0, 0);      // 1
  
   _tstring directory;
  
   directory.resize(spaceRequired - 1);     // 2
  
   const DWORD spaceUsed = 1 + ::GetTempPath(
      spaceRequired,
      const_cast<tchar*>(directory.c_str()));    // 3
  
   if (spaceRequired != spaceUsed)   // 4
   {
      throw CException(_T("GetTempPath()"),
         _T("Failed to get temp path"));
   }
  
   return directory;
}

First I called the function to determine the required size, note that the required "size is the size of the buffer in TCHARs" needed, which includes the terminating null character. Next I created a string that was the correct size (note that I'm removing 1 because the required size includes space for the null and I'm resizing the data area of the string and the null is added on for me. Thirdly I call the API again with my buffer. I then check that the second call uses all of the space that I allocated in the first call and fail if it doesn't.

There's quite a lot wrong with this code but mostly it will work... The check at stage 4 is overly restrictive but shouldn't pose a problem, except that it does... If your temp path contains an unnecessary double backslash; C:\\temp for example rather that C:\temp then the first call will return the length including the double backslash and the second call will return the path with the unnecessary backslash removed. This means that for the example path shown above, the second call returns spaceRequired - 2 rather than the expected spaceRequired - 1 and the code fails with the rather useless error message.

The fixed version of the code doesn't suffer from this problem.

_tstring GetTempPath()
{
   const DWORD spaceRequired = ::GetTempPath(0, 0);
  
   if (spaceRequired == 0)
   {
      const DWORD lastError = GetLastError();
  
      throw CWin32Exception(_T("GetTempPath()"), lastError);
   }
  
   _tstring directory;
  
   directory.resize(spaceRequired - 1);
  
   const DWORD spaceUsed = ::GetTempPath(
      spaceRequired,
      const_cast<tchar*>(directory.c_str()));
  
   if (spaceUsed == 0)
   {
      const DWORD lastError = GetLastError();
  
      throw CWin32Exception(_T("GetTempPath()"), lastError);
   }
  
   if (spaceUsed >= spaceRequired)
   {
      throw CException(
         _T("GetTempPath()"), 
         _T("Failed to get temp path, second call needed more space ") +
         ToString(spaceUsed + 1) + 
         _T(" than first call allocated ") +
         ToString(spaceRequired));
   }
  
   directory.resize(spaceUsed);
  
   return directory;
}

Whilst there are still, no doubt, style issues, this version checks for errors in a better way (I'd first thought that the failure was some kind of permissions thing) and reports these errors in a clearer manner. Hopefully it will be another 10 years at least before I get another error report for this piece of code.