Rambling Comments

Why not to compile as 64 bit...

2009-06-11T13:06:55Z

Here's a nice piece by Rico Mariani about why Visual Studio is unlikely to go 64 bit any time soon. In a nut shell, unless you have very large data sets that need to be kept in memory you might be worse off as a 64 bit process than you are as a 32 bit process on a 64 bit operating system. Food for thought.

Summary of x64 Win32 Debug API issues

2007-10-04T07:41:35Z

I've finished porting my debugging tools support libraries to x64 now and thought it was worth putting up a summary of the issues that I've noticed:

A 32bit exe can't start a 64bit exe for debugging - pretty obvious really.

When a 64bit debugger is running a 32bit debugee the debugger seems to get TWO "loader breakpoints" one when the 64bit dlls are loaded and a second when the 32bit dlls have been loaded.

In a 32bit debugger running on WOW64 a call to DebugSetProcessKillOnExit() succeeds but doesn't seem to do what it's supposed to do.

When a CLR process is started on an x64 machine you can't rely on the image type of the stub to tell you if it will be a native x64 process or an x86 process; the stub for a .Net 2.0 process can be x86 yet can still cause a native x64 CLR host to run the app...

When running a CLR app under the Win32 debug interface you only ever seem to hit the native entry point if you're running under WOW64. In all other situations you don't hit the native entry point ever. If you rely on it to pause your debug tools once the process is completely loaded and ready to roll then you need to stick a break point in _CorExeMain in mscoree.dll. What's more, if you're on x64 you might not even be able to access the native entry point's memory...

When injecting a dll into a process using the "Richter" thread calls LoadLibrary() approach remember that a) the dll image type (x86/x64) must match the process image type and b) if you're injecting into a 32bit process from a 64bit process you'll need to enumerate the modules in the 32bit process to get the HMODULE to Kernel32.dll so that you're function pointer to LoadLibrary() is offset correctly...

And now to update the tools themselves so that they run as 32bit processes with 64bit processes stored as resources (along with the 32bit and 64bit injection dlls) so that they can switch over to 64bit if appropriate and can run natively on 32bit and 64bit OSs and inject into either 32bit or 64bit targets...

x64, IOCP, Socket Server framework

2007-10-03T08:18:06Z

The x64 version of our I/O completion port based, networking client and server framework, The Server Framework, is now shipping. This is the licensed version of the free server framework that's available here. An overview of the licensed version can be found here. The changes from that version to the latest version can be found here.

The documentation for this version is available online, here. Please be patient, the docs are a work in progress and I know they could be better.

This release sees a first cut at full documentation for all of the example servers; see here.

Apart from the x64 work this release also includes the changes to our PerfMon library (and therefore supports performance counter installation on Vista, side by side x86 and x64 counter dlls, multiple instances and multiple languages - more details here, docs here.).

There are also some changes to our Service library; you can now install multiple 'instances' of a service (essentially your exe can be configured to run as multiple different services) and you can still run these instances without the SCM in the IDE for debugging by simply specifying a command line switch.

If you want to integrate with .Net then there are CLR hosting samples that allow you to write your business logic in a .Net language and write your networking layer in C++.

WOW64 Win32 DebugAPI and managed code

2007-10-02T08:48:28Z

It seems that I've located the "issues" in my Debug Tools library. This library is used in my TickShifter (time control) tool and my native Win32 Deadlock Detection tool. Due to how I wanted to control the debugged processes start up and how I needed to halt the debugged process at a particular time there were some hoops that I found I had to jump through. Those hoops have changed shape; either because of differences between Vista and XP or due to x64 and the WOW64 layer, I'm not sure which yet.

Anyway, when starting a native process under the Debug API it was always easy enough to set a break point in the process' entry point and halt the debugger at that point so that I could inject code and fiddle around. On XP I found that managed processes never hit the native entry point that's written into the PE file and so I had to spot that the process was managed and shove a break point into _CorExeMain in mscoree.dll as that was the nearest alternative. Under x64 using .Net 2.0 I still have to do this (and what's more the entry point in the PE isn't in writable memory, so perhaps it's an offset to a managed entry point...). However, when running with .Net 1.0 or .Net 1.1 on x64 I end up with the WOW64 layer coming to life (which complicates matters a little) and the native entry point IS hit as you'd expect... An added complication is that I can't just use the code that assumes that the native entry point wont be hit and ignore it if it is because under WOW64 I start getting notifications from other threads before my managed entry point is hit and I need to gain control before other threads are running...

Fun, fun, fun... The good news is that I know what's going on now and I expect I can build in some checks that will work out how to take control of the processes that are being started up. I don't really want to look at the managed debug api just yet as that's not really what I need...

Now I just need to work out how to programatically disable the "program has stopped working, dialog" which pops up when my application under test exits due to an exception as my debugger is in control and I don't want windows trying to do 'clever' things for me... And, of course, it's all different under a Win32 build on x64...

x64 Debugger, ExceptionCode == 0x4000001f

2007-10-01T12:50:33Z

I should be finishing some docs for the x64 release of The Server Framework... But this is more interesting...

When running my Win32 debugging code on x64, this time when compiled natively as x64 code and when debugging an x64 CLR process, I've been getting an 'unexpected' ExceptionCode in an EXCEPTION_DEBUG_EVENT. The code is 0x4000001f and, after some searching around, it seems that it's a STATUS_WX86_BREAKPOINT event and I sometimes get these instead of EXCEPTION_BREAKPOINT events...

This is a bit of a surprise, some breakpoints that I've set generate the expected debug event and some generate the 0x4000001f one... This isn't too bad to handle in code except for the fact that I'm getting one of these debug events generated for a break point that I havent set and know nothing about...

DebugSetProcessKillOnExit and Win32 processes on x64

2007-10-01T09:55:03Z

I spent a little time looking at an x64 port of my debugging tools library at the weekend. Since this requires me to set breakpoints and manipulate process memory and image files and all sorts I expected it to be a little more complex to port than the higher level sockets code and the bulk of my Win32 code. So far things are going reasonably well, but I've just come across a strangeness with DebugSetProcessKillOnExit(TRUE) in an x86 debugger that's running an x86 process on an x64 machine. It seems like when the debug thread terminates the debugged process doesn't get terminated, it should do, but it doesn't... :(

What's more, changing my code to terminate the process forcibly, with TerminateProcess() if my wait for the debugged process to terminate cleanly times out also fails... The call to TerminateProcess() returns without error but the process is still alive and remains that way until the debugger process itself terminates...

My current best guess is that my code is subtly broken and there's a race condition being exposed on my x64 box that never shows up (the tests don't fail) on my x86 box...

How to liquidise perfmon's guts but still leave the body standing...

2007-07-19T08:22:16Z

I'm still working on my performance counter library. I've got parent and child objects working and multiple instances and I have a sample server that publishes performance data using two objects, one for the executable (io threads, sockets and buffers in use, etc) and one for each of the server instances within the executable (connections active, bytes/sec, etc). The per server instance counters have the server executable as a parent and it has some totaled up counters for each instance... In turn there's a _Total instance in the server executable object's instances and that totals all executables... You can run multiple servers and they all just plug themselves in to the system and show their counters. It's nice and provides the functionality that one of my clients might need, which is handy, and exposes functionality that should have been exposed anyway...

Once I got all of the above working on my x64 dev box I decided to try a build on my older 32-bit XP box. Everything was good (once I dealt with the fact that the InterlockedIncrement64() family of functions exist in the headers but not in Kernel32.lib on the 32-bit XP box and so I was getting a run-time failure). However, registering the counters caused perfmon on the XP box to hang, it got to the point where it should display counters and just stopped. This was the problem that I was having with the side by side installation of 32-bit and 64-bit counter dlls on the Vista machine... What was strange was the the calls to OutputDebugString() that I'd put into the performance counter dll's Open() call were working, the dll was being opened, the call to open was returning without error and then, at some point after that, perfmon simply died inside. The shell and GUI remained somewhat responsive for a period of time but there was no monitoring going on.

I shut down my boxes, wandered off to the station to meet Gaelle after her skate and let the problem float around in my head looking for a solution... Sometime later I was just about to drift off to sleep when I realised what the issue was. Calling convention. Way back at the start of the project I'd had some issues with getting the counter dll exports working and had changed the calling convention to explicitly be __cdecl. This worked fine, on x64 (because all of the calling convention compiler modifiers are ignored on x64 and there's only one calling convention!) and looked like it worked back on Win32, well it gave the right entry points... However, a moment's thought (or a quick look at this dummy's guide to win32 calling conventions) will reveal that if the calling code expects a function to use __stdcall and the called code is actually using __cdecl then Bad Things will happen when the stack is cleaned up by both parties... Of course the name mangling is supposed to be different to help protect you from this, but, well...

So, this morning I adjusted the code so that it built as __stdcall, well, WINAPI actually, on both platforms and then used some #pragma magic to rename the exports to the desired names on Win32. The result is that I now have side-by-side performance counters working on x64 and the 32-bit code works on XP! Woo!

Tasks for today include writing a test to prevent the calling convention issue biting me again with this code and then adding multi-language counter and help string support. Once that's done it's pretty much all over bar the refactoring.

Performance counter rewrite almost complete

2007-07-17T16:54:00Z

Around a week ago I started looking at rewriting my performance counter library so that it worked on Vista. That work is almost complete and I'm at the stage where the new library does everything the old one did but there's a list of 'nice to have' new items that I haven't completed yet. The work was reasonably easy once I got into it. I wasted an afternoon or so trying to understand the original code that I'd hacked a little way back in 2000 and eventually gave up and went to the documentation (and the book from whence the original code came). It's not that the original code was bad in any way, indeed it was clever and tight and worked well, it's just that it was a little obtuse. I'm not convinced that my new code is any less obtuse but at least I've split the code into several classes which just do one thing rather than having a single class with parts of it that are compiled out depending on whether you're building the application that logs data or the dll that collects it... I also wrote the whole thing in a test driven manner, which means I have tests for all of it, which is nice and will make the next step, refactoring it a little, easier.]]> My new design has, at its core, a "data block" class which manages the shared memory and associated interprocess locking that's needed to communicate between the application(s) that are logging and the dll that's collecting. This is where the first of the improvements comes in; the old code always allowed multiple instances of a performance object but they weren't especially easy to manage. The new code makes this much easier. It's now possible for applications to dynamically add and remove counter instances and for multiple applications to connect to different instances of the same counter set. This means that there'll soon be an example server that logs performance information per server instance within a server process and across server processes - which a client currently requires.

The next class is a schema class. This allows you to define your performance objects and replaces the macros and clever compile time construction of the performance objects that the old code allowed. There are some TODO's for the schema class, I'd like to be able to save the structure to a resource in the exe and dll so that you dont need to explicitly build the schema in code in both of them (it's not so terrible to do so, you simply include your custom schema class in each project) but it would make the collection dll's slightly smaller and more efficient if they didn't need to pull in the STL just because of the schema.

Then we have a monitoring class for use by the application, this provides access to the performance objects, instances and their counters. I'm pleased that this class basically just helps you map from the schema to the correct memory blobs in the data block so that actually incrementing a counter is still simply a case of doing an InterlockedIncrement() on a block of shared memory.

Then there's a collector class which is used by the collection dll. This works with the data block to format and return the performance data in the correct way. It doesn't need to do a great deal as most of the data is formatted in the right way in the data block anyway...

Finally there's an installer class that takes a schema and writes out the appropriate registry entries and calls the appropriate APIs to register your counter DLL. This uses a 'file exporter' that takes the schema and writes out an appropriately formatted .h and .ini file into a temporary directory and so that we can call LoadPerfCounterTextStrings(). The installer makes it easy for us to load localised strings for the counter names and help, so there's an open TODO item on the schema class to support this.

I'd like to be able to say that since I wrote all of the code in a test driven manner that when I came to install a performance counter DLL and look at the results in perfmon it just worked. It didn't. The tests made sure that the code did what I understood the spec to be. My understanding was slightly flawed in places and so I needed to do some debugging to get the counters to work. The debugging was pretty easy since the resulting changes were supported by the tests that I had. As I brought my understanding of the spec in line with what the spec actually said and adjusted my tests to suit it was easy to then adjust the code so that the tests passed again. So, next on the list is an example server to show off the mutliple instances stuff and then one that uses multiple languages in the counters.

My only currently outstanding "strangeness" is how to install the 64bit and 32bit counter dlls so that I can run a 64bit version of perfmon locally and a 32bit version remotely... Putting the dlls in System32 and SysWOW64 doesn't seem to be enough...

x64 port - a diversion

2007-06-14T11:44:45Z

The x64 port of The Server Framework is going well. I've run the tests for the OpenSSL servers and things are looking good there; this is always an important stage to get to for me as these servers and their tests really thrash the framework and are good at shaking out any bugs that might be hiding...

So, now I'm faced with the monkey work of adjusting all of the other example servers to build cleanly on all platforms and (in addition to the x64 work) to support either IPv4 or IPv6... My problem is that I now support 5 compilers; VC6 through VS2005 + x64 and, since both I and my clients like being able to work in the IDEs, this means 4 project files to maintain per example. Add to this the fact that when I test the example servers I build and run each of the 5 build configurations on each of the 5 compilers and you can start to realise the amount of work that's ahead... Then when you add in the fact that on most compiler/platform combinations I support either the built in STL or STLPort (various flavours) and also like to be able to build against various flavours of Platform SDK and, well, it's a lot of work...

I'm currently thinking about how to make this a bit easier to manage. I'm switching to VS2005 as my 'every day' compiler which means I need to either manually maintain the other compiler's project files or finally write the converter that I've been thinking about ... Each of the server tests assumes it's the only test running on the machine at the time (they all use the same port) and, besides, they thrash the box... But... I now have two dev boxes and a couple of laptops so a distributed build and test system would be nice, perhaps using VMWare...

So, right now, I'm thinking, and I'm sure that it's best to solve these problems now, rather than plugging on with the monkey work by hand... So, if you're in desperate need for the x64 version of The Server Framework then get in touch and I'm sure I can put together an interim release for you, but otherwise the official release date is likely to be some time from now...

x64 Socket Server framework

2007-06-07T16:27:34Z

My new development box arrived last week. It's a bit of a monster, dual processor (Intel Xeon E5320 quad core, 1.86 GHz), 8GB ram, and it runs 64bit Vista very nicely indeed. Since it's an x64 box I have started work on porting The Server Framework to x64. So far things are going very smoothly. The main issue is that I have, in the past, tended to overuse size_t when I really mean "a reasonable sized unsigned value". Since size_t doesn't really fit the bill for that on x64 most of the changes I've had to make are to reduce size_t counts and lengths to DWORDs. In all honesty, in a lot of cases, they should always have been DWORDs...

I have the framework compiling cleanly and the first of the example servers running nicely, but there's a long way to go until I can ship this version to clients. I have to make sure that all the other libraries of code that depend on the socket tools library and the libraries that it depends on are also clean on x64 and then all the tests have to build and run and pass on all compilers and platforms... As I said when I finished the last major changes to the framework, it's about time I automated all my build and test scenarios...