« CORBA | Debugging Tools Archives | Dumbing down is dumb »

TIME_WAIT perfmon counters

| 4 Comments
I've built a small Windows Service which exposes perfmon counters to track sockets in TIME_WAIT state. It can be downloaded from the links later in this post. Back in 2011 I was helping a client look for issues in their systems caused by having too many sockets in a TIME_WAIT state (see here for why this can be a problem). This was affecting their connectivity. Rather surprisingly there seemed to be no way to track the number of sockets in TIME_WAIT using perfmon as there didn't seem to be a counter exposed. A counter would have been useful so…

Alternative call stack capturing

| 2 Comments
I've just stumbled on these blog posts, by Maciej Sinilo, a game developer. He's written a memory allocation monitoring tool and mentions that using RtlCaptureStackBackTrace() is a faster (if undocumented) way to capture a call stack. This is interesting to me as the call stack capture code in my debugging tools (deadlock detection, timeshifter, tickshifter, etc.) is pretty slow when using StackWalk64(). It's also interesting that he seems to store and sort stacks by CRC which is similar to what I do in my tools. This entry's really just so that I don't forget where I found the link... Oh,…

More on the CLR startup change

| 0 Comments
Last week I mentioned that some of my tests for my Win32 Debug API class had suddenly started failing. It seems that I was right and the changes are due to some .Net fixes that have been rolled out recently. The code runs and the tests pass if I run on a clean instal Vista x64 VM and fail on my day to day development box. It seems that my plan to "stick a breakpoint in mscoree.dll's _CorExeMain()" wasn't such a good idea after all. With the new updates installed and with an x64 process using the Win32 debug API…
Back in October 2007 I sumarised my findings from getting my Win32 DebugAPI based debug engine working on x64. One of the strange things that I found at the time was this: When running a CLR app under the Win32 debug interface you only ever seem to hit the native entry point if you're running under WOW64. In all other situations you don't hit the native entry point ever. If you rely on it to pause your debug tools once the process is completely loaded and ready to roll then you need to stick a break point in _CorExeMain in…

Summary of x64 Win32 Debug API issues

| 0 Comments
I've finished porting my debugging tools support libraries to x64 now and thought it was worth putting up a summary of the issues that I've noticed: A 32bit exe can't start a 64bit exe for debugging - pretty obvious really. When a 64bit debugger is running a 32bit debugee the debugger seems to get TWO "loader breakpoints" one when the 64bit dlls are loaded and a second when the 32bit dlls have been loaded. In a 32bit debugger running on WOW64 a call to DebugSetProcessKillOnExit() succeeds but doesn't seem to do what it's supposed to do. When a CLR process…

WOW64 Win32 DebugAPI and managed code

| 0 Comments
It seems that I've located the "issues" in my Debug Tools library. This library is used in my TickShifter (time control) tool and my native Win32 Deadlock Detection tool. Due to how I wanted to control the debugged processes start up and how I needed to halt the debugged process at a particular time there were some hoops that I found I had to jump through. Those hoops have changed shape; either because of differences between Vista and XP or due to x64 and the WOW64 layer, I'm not sure which yet. Anyway, when starting a native process under the…

x64 Debugger, ExceptionCode == 0x4000001f

| 1 Comment
I should be finishing some docs for the x64 release of The Server Framework... But this is more interesting... When running my Win32 debugging code on x64, this time when compiled natively as x64 code and when debugging an x64 CLR process, I've been getting an 'unexpected' ExceptionCode in an EXCEPTION_DEBUG_EVENT. The code is 0x4000001f and, after some searching around, it seems that it's a STATUS_WX86_BREAKPOINT event and I sometimes get these instead of EXCEPTION_BREAKPOINT events... This is a bit of a surprise, some breakpoints that I've set generate the expected debug event and some generate the 0x4000001f one... This isn't too…
I spent a little time looking at an x64 port of my debugging tools library at the weekend. Since this requires me to set breakpoints and manipulate process memory and image files and all sorts I expected it to be a little more complex to port than the higher level sockets code and the bulk of my Win32 code. So far things are going reasonably well, but I've just come across a strangeness with DebugSetProcessKillOnExit(TRUE) in an x86 debugger that's running an x86 process on an x64 machine. It seems like when the debug thread terminates the debugged process doesn't…
Back in April 2006 I posted a copy of TickShifter, see here for details. It seems that there was a bug in my Win32 debugger code on which TickShifter is built. The bug was that we failed to "forget about" dlls that were unloaded... Because we failed to forget about them it was possible for the debugger code to try and do something with addresses in these dlls that were no longer loaded and this would cause a C++ exception on the debugger thread when our call to ReadProcessMemory() failed and this caused all sorts of problems... The end result…
It seems that Vista contains lots of interesting new Win32 API calls and some of these provide built in support for deadlock detection... I guess my deadlock detection tool can operate differently on Vista then...…

Mock32, parameterize from below

| 7 Comments
It's funny how potential product ideas beget other potential product ideas and the thing that you eventually end up with as a product for sale is often far from the original product idea and the code that you started to write... I'm not even there yet and I'm on my third wave of product idea focus. Right now I'm working towards getting some developer tools out there to help people write better systems. This is the API hooking stuff that I've blathered on about in the past. The original product idea that started it all was a niche market testing…
A few weeks ago I wrote about hijacking threads. The principle was good, the code worked but the reason I needed it was flawed so the exercise was a failure. I assumed that it was likely to be a failure when I set out though, so it's more a confirmation that I need to do things properly rather than hackily... The idea was the try and outsmart the Windows program loader. My API hooking tools currently hook the target process when it gets to a point where it has started up but no 'user' code has run. Well, that's not…

Hijacking a thread in another process

| 2 Comments
My API hooking currently relies on creating a block of memory in the target process to hold the string that you need to pass to LoadLibraryW() and then calling CreateRemoteThread() and passing the address of LoadLibraryW() as the thread start address and the address of our block of memory as the thread parameter. It works well and even allows you to know if the operation worked, by waiting for the remote thread to terminate and then examining its exit code. However, there are times when you might not want (or be able to) create a thread in the target process…

I love it when a plan comes together

| 1 Comment
Discovering and then fixing the problem that prevented my tools from running on themselves planted a seed of an idea in my mind. My APIHook library and any code that used it died horribly under leak testing tools such as Purify and BoundsChecker. I'd put it down to me trying to be too clever but not being clever enough and ignored it but it's a pain to have code that you can't polish. Solving the problem of the "wrong address" suggested to me that perhaps these other tools were causing similar problems. It's most likely that they both operate in…

Tools, debug thyselves

| 0 Comments
One of the first things that I tried to do with the latest release of my TickShifter tool was to run it on itself. Of course, that didn't work. There are several reasons why the tool might have had problems running on itself but I thought I'd addressed most of them when I dealt with the issues around getting multiple copies of the tool running at the same time. The problems there were mostly around the names that I used for the named kernel objects that were required; the control program communicates with the dll that it injects so that…

TickShifter v0.2

| 0 Comments
As I mentioned a while back, I've been working on adding rudimentary GUIs to my debugging and testing tools. In fact, both the deadlock detection tool and the time shifting tool are both functional enough for real world use but the command line interfaces make them harder to use than they need to be. I'm not yet 100% sure of the form that the final GUI should take, but I've pressed on with my initial GUI development to produce a reasonably complete GUI for the cut down, demonstration tool, TickShifter.…

TickShifter v0.1

| 0 Comments
Well, I figure that I've written about these debug tools that I've been working on for long enough. The time has come to make one available for other people to use and abuse. Given that I hope to sell some of these tools eventually I've decided not to give away early versions at present. Much as I will need feedback I think a properly structured beta program would be better for that. So, what I have done is write a cut down tool that demonstrates and uses the same technology that the other tools use. In essence if this runs…

Deadlock detection tool updates

| 0 Comments
When I came back from skiing in Colorado I had a bug report from a client and it took me a fair while to isolate the problem for them. The report suggested that a server that I'd built for them a while back was deadlocking in certain situations. The server had always had deadlock problems due to a couple of poor design decisions that I made early on and I'd built a tool to help me remove the potential for deadlock from it by analysing its lock acquisition patterns - the code was too complex for me to do this…

When your mocks are executable

| 0 Comments
The size of the "units" that I test with my unit tests varies quite considerably. Usually I tend to work at the single class level; the test will provide mocks for the services that the class requires and the tests will ensure that the class behaves correctly and interacts as expected with the its service providers. Sometimes though the units consist of a whole subsystem, still often a single class, but one which wires up and collects many other classes into a single component; again the service providers that it uses may be mocked but they're often only right on…

Purecall

| 2 Comments
I've been plugging away at my multi-process Win32 debugger code this week and one of my test harnesses had started to suffer from intermittent "R6025 - pure virtual function call" errors. These tend to crop up if you call a virtual function from a constructor or destructor or if there's a race condition between calling a method on an object from one thread and destroying the object in another thread. For me, at least, they tend to be simple mistakes to fix once I know where the problem lies. Unfortunately, often the standard error report of a "purecall" is less…

Beware the momentum of prototype code

| 4 Comments
A while back Chris Baus wrote a little rant about prototypes. I started to write a piece that defended prototypes when correctly used and ran out of steam; or, more to the point, couldn't really say very much except they're OK except when they're not, try not to get to the point where they're not. Having just recently been bitten by a prototype that was trying to rise above its station, I now have more to say.…
There's an interesting article by Tomer Abramson in this month's Dr Dobb's Journal about deadlock detection. He provides a compile in tool that works in a similar way to my deadlock detection tool and reports on potential deadlocks in code even if they never occur during the program run in question. His architecture is considerably different to mine but the idea is the same. By recording the sequence of lock acquisition and release on all threads in the program you can then examine the sequence that locks are taken out and spot potential deadlocks.…

More thoughts on thread naming

| 5 Comments
I've just finished the first cut of the code that handles the Win32 Thread Naming "API". This means that my debugging tools can now report the name of a thread (if it has been named) rather than just its thread ID. Having implemented the code in the debugger to deal with the exception I find myself wondering about API design and why the original designer of this particular facility decided to make it hard on themselves.…

Naming Win32 threads - how it works

| 0 Comments
I've been using the SetThreadName() function from the MSDN docs for ages to provide a way of distinguishing between threads in the VC debugger. I always thought it was a bit of a weird 'API' but had never really thought to wonder why. Steve over at "Bug Babble" explains how the 'API' works from the debugger's point of view and, well, then it all becomes obvious. For some reason I'd always thought that the 'API' was communicating with the OS in some weird and wacky manner; I hadn't twigged that in fact it was just communicating with the debugger using…

I think the weirdness was me

| 0 Comments
A couple of days ago I mentioned that I was having some problems with loading symbols for a common controls dll. I'm now pretty sure that it was my problem, as usual, rather than someone elses. I've reworked my process startup controlling code and now set and clear the breakpoints in a more reliable (i.e. less hacky) way and things are behaving themselves again. I think I was failing to stop one of the threads properly and that was later causing problems.…

DbgHelp weirdness

| 0 Comments
I was using one of my home made debugging tools recently and it kept crashing :( I assumed it was something I was doing but I've eventually tracked it down to where I load the symbols for the loaded modules in the target process. Attempting to load the symbols for x86_Microsoft.Windows.Common-Controls_6595b64144ccf1df_6.0.2600.2180_x-ww_a84f1ff9\\comctl32.dll leads to dbghelp.dll generating a BEX event (which appears to be either buffer overrun or data execution prevention related).…
Every now and then I come across a situation where encapsulation has been taken slightly too far. Usually, or at least most recently, these over encapsulated designs have had problems because they've blocked access to an event handle. It's great to wrap up a bunch of Win32 primitives into a nice coherent class but if you expose a method that allows the user of the class to wait for something to happen then it's probably also a good idea to expose a handle they can wait on as well. Failure to do so imposes unnecessary restrictions on how the user…

Controlling Time, Take 2

| 5 Comments
Recently I finished developing a high performance ISO-8583 financial transaction authorisation server for a client using The Server Framework and whilst I was running the final black-box tests against the server I realised that these particular tests were dependant on the system date of the machine running the server. The server uses the current time and date to make some decisions on whether it can authorise a particular transaction or not. The date is used to look up some information from a database table. The table contains details of whether a particular competition type is available for entry or not; it has…
It seems to be stack walking week; what with Ned's posting earlier about walking an unmanaged stack and now this excellent piece by David Broman on doing the same in with managed code.…

More on locking

| 0 Comments
Jeff Darcy over at "Canned Platypus" writes about "How to add locking to a program". He laments the lack of a reasonably priced deadlock detection tool. I assume, from his backgrond, that he's interested in an Linux tool, so my deadlock detection tool wont help him much but it's good to know that it's not just me that thinks such a tool would be useful... Jeff links to some expensive static code analysis tools that do deadlock detection. My tool doesn't rely on static code analysis, so you don't need to have source to run it. It can tell you…

Walking the call stack

| 0 Comments
Ned Batchelder has written about the code he uses to get a call stack out of a windows program (thanks for the link Barry). I've added a snippet of the code I use as a comment to his post. Note: the deadlock detector mentioned in this blog post is now available for download from www.lockexplorer.com. I started looking into working with windows call stacks a while ago when I was working on my deadlock detection tool. What surprised me was how easy it was to get a call stack once you understood the DebugHelp API. There are lots of examples…

In summary, don't summarise too soon

| 0 Comments
I've been working on my deadlock detection and lock monitoring tool quite a lot this week; that and fixing the issues that it's been highlighting. Yesterday I decided that I was drowning in data and that I really needed a GUI and, as I thought about how the GUI should work, I realised that I really didn't have enough data. Note: the deadlock detector mentioned in this blog post is now available for download from www.lockexplorer.com.…

What's with CreateProcess and managed exes?

| 1 Comment
I have some code that launches another process in a suspended state, does some stuff (tm) and then resumes the process. This uses CreateProcess() and works fine on unmanaged exes. When I try and launch a manged exe with the same code the process starts up in a running state rather than in a suspended state. The docs for CreateProcess() don't mention any strangeness. Does anyone know of any docs that explain what's going on?…

Viewing lock lifetimes

| 0 Comments
I added some more monitoring to the deadlock tool. You can now view the entire life cycle of the locks, from initialisation, through usage to destruction. The lock usage figures put the contention figures in context as you can see how often the lock was acquired by each thread and compare that to how often there was contention... Note: the deadlock detector mentioned in this blog post is now available for download from www.lockexplorer.com.…

Observing lock contention

| 0 Comments
The deadlock detection tool that I was working on last week is coming on nicely. I managed to locate two potential deadlocks that had never caused any problems in code that is running live on a client's site. Once I had a report from my tool it was relatively easy to fix the problems. Both of these were real bugs that just hadn't bitten yet. Today I added some rudimentary lock contention monitoring to the tool and the results seem to be quite useful... Note: the deadlock detector mentioned in this blog post is now available for download from www.lockexplorer.com.…

I've been lazy this week

| 4 Comments
As I mentioned in an earlier posting I've been working on a tool this week. I'm too lazy to do a job manually and so I decided to write a tool to help me do it... Note: the deadlock detector mentioned in this blog post is now available for download from www.lockexplorer.com. The tool is designed to help me track down deadlocks in code. I decided I needed this tool because I wrote a piece about debugging deadlocks in Visual C++ and realised that using trial and error to locate deadlocks in some client code simply wasn't good enough. The…
Last week I posted an entry about the simple, hand built, mocks that I use for testing in C++. Whilst I stand by my previous assertion that writing the mocks yourself gives you an further insight into your design from a client perspective I think the discussion that I had with Roy Osherove after last week's posting has made me appreciate the power of having mocks that support the programmatic specification of 'expected' behaviour.…
« CORBA | Debugging Tools Archives | Dumbing down is dumb »

About this Archive

This page is an archive of all entries in the Debugging Tools category.

CORBA is the previous category.

Dumbing down is dumb is the next category.

I usually write about C++ development on Windows platforms, but I often ramble on about other less technical stuff...

Find recent content on the main index or look in the archives to find all content.

I have other blogs...

Subscribe to feed The Server Framework - high performance server development
Subscribe to feed Lock Explorer - deadlock detection and multi-threaded performance tools
Subscribe to feed l'Hexapod - embedded electronics and robotics
Subscribe to feed MegèveSki - skiing