« x64 | Lock Explorer Archives | Winsock Registered I/O »
I've been noticing a strange thing for a while on Windows 8/8.1 and the equivalent server versions. The issue occurs when I'm using a Slim Reader/Writer Lock (SRWL) exclusively in exclusive mode (as a replacement for critical sections). What happens is, when a thread that has just unlocked a SRWL exits cleanly, immediately after unlocking the lock, sometimes threads that are waiting on the lock do not get woken and none of them acquire the lock. At first I spent ages thinking that this was some kind of subtle bug in my LockExplorer tool as initially the problem only manifested…
I've just released new versions of my Lock Explorer tools, LID and LIA. This is quite a big release as it increases the number of locking APIs that the tools instrument from 1 to 3. We now track Slim Reader Writer locks and Mutexes. Arguably the tools should always have tracked these, and possibly more API calls, but the tools have always been first and foremost to assist in the development and testing of The Server Framework and, well, we only use Critical Sections. However, that has recently changed as we're now using Slim Reader Writer locks as a…
I'm currently building a new example server for The Server Framework. This is a variation on one of our proxy server examples for a client that's doing some WebSockets work. The idea is that the server takes an inbound WebSockets connection, creates an outbound TCP connection to the target server and routes data to and from the remote server and the WebSockets client. It's fairly simple stuff to put together once you're up to speed on The Server Framework but my client needed a helping hand and it's another nice example of what you can do with the framework. As I've mentioned before,…
After a week or so of serious dog fooding I've finally got a nice reliable lock inversion detector running as part of my build system for The Server Framework's example servers. Note: the deadlock detector mentioned in this blog post is now available for download from www.lockexplorer.com. The build system has always run a set of 'black box' server tests for each server example as part of the build. These start up the example server in question, run a set of connections against it and shut the server down. This proves that the example servers actually work and has also proved a…
As I mentioned, I've been adjusting my build system and have finally got to the point where my lock inversion detector is suitable to run on all of my example servers during their test phase on the build machines. I'm working my way through the various example server's test scripts and adjusting them so that they use the lock inversion detector, can be easily configured to run the full blown deadlock detector and also can run the servers under the memory profiling test runner that I put together earlier in the week. Note: the deadlock detector mentioned in this blog…

Tangential testing

| 0 Comments
My theorising about the strange memory related failures that I was experiencing with my distributed testing using WinRS have led me to putting together a test runner that can limit the amount of memory available to a process and terminate it if it exceeds the expected amount. With this in place during my server test runs I can spot the kind of memory leak that slipped through the cracks of my testing and made it into release 6.2 of The Server Framework. The idea is that rather than just starting the server we start the server using a monitoring process that first…

Alternative call stack capturing

| 2 Comments
I've just stumbled on these blog posts, by Maciej Sinilo, a game developer. He's written a memory allocation monitoring tool and mentions that using RtlCaptureStackBackTrace() is a faster (if undocumented) way to capture a call stack. This is interesting to me as the call stack capture code in my debugging tools (deadlock detection, timeshifter, tickshifter, etc.) is pretty slow when using StackWalk64(). It's also interesting that he seems to store and sort stacks by CRC which is similar to what I do in my tools. This entry's really just so that I don't forget where I found the link... Oh,…
The free version of the socket server framework contained code that could cause a deadlock during connection closure if you also have a lock in your derived class. There's a lock taken out in CSocketServer::Socket::IsValid() that isn't really required and which can cause a deadlock if you have your own lock in your derived class which you lock in OnConnectionReset() or other server callbacks and which is also locked when you call into the framework via Write() or other calls. The deadlock is due to the lock sequencing; when calling into the framework your lock is locked and then the…

CLR Hosting lifetime issues bite again...

| 0 Comments
I'm looking into adding CLR deadlock detection into the CLR hosting code that's used inside The Server Framework and, once again, the fact that you can't cleanly shutdown the CLR host is causing me problems... Since the CLR can't be stopped by a host without terminating the host process (and that's by design...) you need to be aware that any of the code that you have plugged into the CLR by way of the hosting interfaces can be called into during process shutdown. Since most of the code that you plug in will be in the form of COM objects you can…
It seems that Vista contains lots of interesting new Win32 API calls and some of these provide built in support for deadlock detection... I guess my deadlock detection tool can operate differently on Vista then...…

Deadlock detection tool updates

| 0 Comments
When I came back from skiing in Colorado I had a bug report from a client and it took me a fair while to isolate the problem for them. The report suggested that a server that I'd built for them a while back was deadlocking in certain situations. The server had always had deadlock problems due to a couple of poor design decisions that I made early on and I'd built a tool to help me remove the potential for deadlock from it by analysing its lock acquisition patterns - the code was too complex for me to do this…
There's an interesting article by Tomer Abramson in this month's Dr Dobb's Journal about deadlock detection. He provides a compile in tool that works in a similar way to my deadlock detection tool and reports on potential deadlocks in code even if they never occur during the program run in question. His architecture is considerably different to mine but the idea is the same. By recording the sequence of lock acquisition and release on all threads in the program you can then examine the sequence that locks are taken out and spot potential deadlocks.…

More on locking

| 0 Comments
Jeff Darcy over at "Canned Platypus" writes about "How to add locking to a program". He laments the lack of a reasonably priced deadlock detection tool. I assume, from his backgrond, that he's interested in an Linux tool, so my deadlock detection tool wont help him much but it's good to know that it's not just me that thinks such a tool would be useful... Jeff links to some expensive static code analysis tools that do deadlock detection. My tool doesn't rely on static code analysis, so you don't need to have source to run it. It can tell you…

Walking the call stack

| 0 Comments
Ned Batchelder has written about the code he uses to get a call stack out of a windows program (thanks for the link Barry). I've added a snippet of the code I use as a comment to his post. Note: the deadlock detector mentioned in this blog post is now available for download from www.lockexplorer.com. I started looking into working with windows call stacks a while ago when I was working on my deadlock detection tool. What surprised me was how easy it was to get a call stack once you understood the DebugHelp API. There are lots of examples…

Good stuff

| 4 Comments
I use BlogLines to read my RSS subscriptions. It's pretty good, and now that the performance issues I had initially seem to have gone away, I like it a lot. It's very handy to be able to read my feeds from anywhere and always have them up to date and synchronised. One of the features I like is the little "keep new" check box that each item has; check it and the item stays unread. I use this to keep the interesting stuff on top so that I can write about it later. This week I've been busy with my…

In summary, don't summarise too soon

| 0 Comments
I've been working on my deadlock detection and lock monitoring tool quite a lot this week; that and fixing the issues that it's been highlighting. Yesterday I decided that I was drowning in data and that I really needed a GUI and, as I thought about how the GUI should work, I realised that I really didn't have enough data. Note: the deadlock detector mentioned in this blog post is now available for download from www.lockexplorer.com.…

Viewing lock lifetimes

| 0 Comments
I added some more monitoring to the deadlock tool. You can now view the entire life cycle of the locks, from initialisation, through usage to destruction. The lock usage figures put the contention figures in context as you can see how often the lock was acquired by each thread and compare that to how often there was contention... Note: the deadlock detector mentioned in this blog post is now available for download from www.lockexplorer.com.…

Observing lock contention

| 0 Comments
The deadlock detection tool that I was working on last week is coming on nicely. I managed to locate two potential deadlocks that had never caused any problems in code that is running live on a client's site. Once I had a report from my tool it was relatively easy to fix the problems. Both of these were real bugs that just hadn't bitten yet. Today I added some rudimentary lock contention monitoring to the tool and the results seem to be quite useful... Note: the deadlock detector mentioned in this blog post is now available for download from www.lockexplorer.com.…

I've been lazy this week

| 4 Comments
As I mentioned in an earlier posting I've been working on a tool this week. I'm too lazy to do a job manually and so I decided to write a tool to help me do it... Note: the deadlock detector mentioned in this blog post is now available for download from www.lockexplorer.com. The tool is designed to help me track down deadlocks in code. I decided I needed this tool because I wrote a piece about debugging deadlocks in Visual C++ and realised that using trial and error to locate deadlocks in some client code simply wasn't good enough. The…

Whilst on the subject of deadlocks

| 0 Comments
It must be a deadlock kinda day. Pete McKinstry points to a Java deadlock avoidance scheme which involves knowing and using a total ordering of the locks that you wish to acquire. This is similar to Andrei Alexandrescu's C++ idea of always acquiring multiple locks in increasing memory address order. Both of these are fine if you can get at all of the locks from one place. I find that that's rarely the case and more often the locks are within objects and I don't want to break the encapsulation to expose the need to lock around a method so…
I'm currently adding some functionality to a server that I wrote, using The Server Framework, for a client last year. The server provides application gateway services for them. The new functionality is to switch to using asynchronous connect calls using ConnectEx when it's available on the platform that the server is running on and to continue to use a blocking connect if ConnectEx isn't available. As I mentioned in the posting back in Feb 2004 the performance counters that we add to these kinds of servers are invaluable in tracking down problems. Unusual life-signs indicate internal wierdness ;) Today I finished making…
I've been splunking around Dll loading recently for a pet project. It's been an interesting journey and this evening I solved the final piece of the puzzle and, when I did, I suddenly wondered, not for the first time, why Windows holds the loader lock when calling DllMain()...…

I always regret leaving the perfmon code out

| 0 Comments
I had one of those "Doh!" moments yesterday. In summary, always put the performance monitoring code in early, looking at a program's vital signs as a jiggly graph can show up all kinds of unexpected things...…
« x64 | Lock Explorer Archives | Winsock Registered I/O »

About this Archive

This page is an archive of all entries in the Lock Explorer category.

x64 is the previous category.

Winsock Registered I/O is the next category.

I usually write about C++ development on Windows platforms, but I often ramble on about other less technical stuff...

Find recent content on the main index or look in the archives to find all content.

I have other blogs...

Subscribe to feed The Server Framework - high performance server development
Subscribe to feed Lock Explorer - deadlock detection and multi-threaded performance tools
Subscribe to feed l'Hexapod - embedded electronics and robotics
Subscribe to feed MegèveSki - skiing