#pragma unmanaged
I’ve just spent a little too long trying to track down a bug in a mixed mode DLL that I’m building. The DLL exposes a set of entry point functions that are defined as taking a single pointer argument and lies to the application that hosts it so that the application can call it with various numbers of arguments. The arguments could change from call to call or from ‘session’ to ‘session’. This all works fine thanks to the wonders of __stdcall and the fact that my DLL knows what arguments to expect and so can unmarshall them correctly by working from offsets from the single known argument that the entry point functions actually take. All of this has been working for ages but I’ve recently begun adjusting the code to make it more flexible with regards to the hosts that it supports and in the process of adjusting it the parameter marshalling code stopped working.
What began happening was that the first argument could be accessed fine and all of the other arguments simply didn’t seem to exist. Whereas I could previously walk along my ‘argument format string’ and then step along memory from the address of the first argument and locate and decode all subsequent arguments now I just got what looked like random memory…
The problem was that the entry points into the DLL were now being compiled as managed code rather than as unmanaged code. This was something that took me far too long to work out, but once I did work it out it was obvious (if I scrolled down the call stack far enough I could see an unmanaged to managed code transition outside of my DLL before my entry point was hit). This transition did some clever managed stuff to the arguments being passed in and rendered my “offsets from arg1” method of decoding useless.
Luckily once I realised what was going on I simply specified that the entry point was unmanaged (#pragma unmanaged) and everything started working again… There’s now a nice big comment in the entry point’s file explaining what the problem is if I ever see the symptoms that I was seeing. Hopefully it will prevent some wasted time if it happens again…
