An overview of features of the licensed Server Framework

The licensed version of my freely available I/O Completion Port based, high performance, Windows networking framework provides a whole host of features that the free code doesn’t; aside from performance improvements, bug fixes and an active support and development process. These features make writing highly scalable TCP and UDP clients and servers very easy and solve many of the problems that you would come across if you were to start from scratch, either with the Winsock API directly or using something such as boost::asio. Whilst my client/server framework deliberately does not offer cross platform functionality it targets the writing of high performance clients and servers on Windows and doesn’t sacrifice performance for ease of portability. It uses I/O Completion Ports to provide highly scalable asynchronous I/O and has been designed from the ground up to be configurable and ‘pluggable’; it’s easy to replace functionality by simply providing your own implementation of a standard framework interface. The framework comes with full source code, a simple and straight forward licensing arrangement, free support and updates for life and numerous fully working demonstration clients and servers that help to explain how various aspects of the framework work and provide you with a base for building your own clients and servers. Anyway, here are some of the things that you get ‘for free’ when using the licensed code.

• Connection limiting - strangely enough, one of the most important things that you need when you’re writing highly scalable servers is a way to limit the scalability when needed. Since the framework code itself wont impede your ability to deal with 10’s of thousands of concurrent connections it’s sometimes necessary to place an explicit restriction on the number of connections you will actually allow. The main reason for this is that each of your connections will use resources, be it memory, non paged pool, database connections, bandwidth, whatever. Before moving into production it useful to profile how your server operates on the hardware that you’ll be using and then limit the number of connections that it can process to a number that allows it to operate safely within the hardware limitations. Other clients find that the ability to limit connections is useful to them for licensing their resulting products based on a X connections for $Y based licensing scheme. As with most aspects of the framework, connection limiting is ‘pluggable’ you can either use a standard implementation of a connection limiter or implement your own by creating a class that implements the ILimitConnections interface.

• Asynchronous read timeouts - often you’ll find that your client or server needs to act on inactivity. In an interactive server where you have a physical human being at the end of a connection you may need to close down connections when they haven’t been used for a while. Unfortunately Winsock doesn’t provide a way to set timeouts on asynchronous reads; SO_RCVTIMEO simply doesn’t work on a non-blocking socket. The framework provides a very easy way to deal with this Winsock limitation in the form of the CReadTimeoutStreamSocketConnectionFilter class which comes with a complete sample server that demonstrates its usage.

• Flow Control - to be able to send as much data as possible over a connection using asynchronous I/O you need to take an active interest in managing flow control between the peers involved. Failure to do this can result in unconstrained resource usage either on the client or the server as you continue to try and send data whilst the TCP stack’s transmission window is full. The CFlowControlStreamSocketConnectionFilter provides a pluggable way to actively manage flow control over a TCP connection and comes with fully functional demonstration clients and servers which implement a high performance data distribution server using TCP.

• Automatic connection re-establishment - some clients and servers need to re-establish failed connections, possibly waiting a predetermined period of time before they retry the connection. The CConnectionMaintainingStreamSocketConnectionFilter class provides this functionality automatically for you with configurable connection re-establishment timeouts and the ability to selectively enable the re-establishment at the individual connection level.

• Compression - it’s often desirable to compress a TCP stream to reduce the amount of data that needs to be sent. The CCompressingStreamSocketConnectionFilter (which will be available with release 6.2 of the framework in early 2010) provides ZLib compression for your TCP connections automatically. You don’t need to worry about compressing or decompressing the data and the filter can be added to existing servers and clients with minimal code changes.

• Stream Filtering - all of the filters mentioned above are implemented using the filtering API which is part of the framework. This API makes the whole ‘stream filtering’ process available for your own plugins and the source code for the numerous filters that are supplied as part of the framework makes it easy to implement your own. Filters can be “stacked”, so that it’s easy to compose your server from existing functionality; you may have an SSL protected compressed TCP stream with explicit flow control, read timeouts and connection re-establishment simply by pushing the appropriate filters onto your stream. All filtering occurs before your own code gets to see the data so that you don’t need to alter your business logic if you suddenly need to add compression, or security to a connection.

• Server collections - often you may need to build a server which listens on multiple ports to provide its functionality, or one that uses UDP and TCP or that provides both cleartext and encrypted connections on different ports. The CServerCollection and CNamedServerCollection classes provide an easy way to manage groups of server objects, allowing you to start, stop, and pause all or some of the servers easily. Of course, multiple server objects can share socket allocators, buffer allocators and the thread pool used for I/O operations to share the resources required for high performance.

• Efficient data broadcasting and scatter/gather I/O - the framework supports partial buffer reuse, buffer broadcasting and scatter/gather I/O with various buffer allocators. When broadcasting data or when building a response from standard parts and request specific parts the framework is optimised to provide you with the tools you need to be able to send data to multiple clients with the minimum of memory copying.

• Page aligned buffers - when striving for the ultimate performance it’s often desirable to use page-aligned buffers for your socket I/O as this can help to restrict the number of memory pages that the operating system needs to lock in memory during data transfers. The standard framework buffer allocator, CBufferAllocator, has options that allow it to produce page-aligned buffers if you need it to.

• Connection collections - clients and servers often need to be able to deal with all, or a subset, of their connections from the context of any other connection. Chat servers may need to allow one client to send data to another, as may data distribution servers. The framework provides several ‘connection collection’ classes which make it easy to manage a collection of connections. The CStreamSockeConnectionCollection class deals with efficiently collecting connections into a grouping that you can search or iterate. The slightly more advanced CStreamSocketBroadcastableConnectionCollection class adds the ability to efficiently broadcast data to multiple connections without duplicating it. More about these classes can be found here.

• Monitoring - when striving to build a high performance system one of the most important things is the ability to measure the effect that your changes have. The framework contains many monitoring interfaces that allow you to simply plug a monitor into the heart of the code that runs your networking. These monitoring interfaces can then be hooked up to your preferred monitoring solution; I tend to go with ‘perfmon’ counters. This allows you to track and visualise the effects that your changes have during development and to monitor your product once it’s in production.

• Lightweight timers - complex servers can often require several independent timers per connection, the lightweight timer implementation that ships with the framework makes it easy to manage 10’s of thousands of timers. Since timers and periodic scheduling are such a common issue there’s an easy to follow “How to” available that explains how the supplied demo servers use timers.

• User data - it’s all very well having a server that can deal with 70,000 concurrent connections but what happens when you need to store data with each connection, looking up that data in a collection keyed on the connection identifier can ruin performance due to the locking required. Luckily the design of the framework allows you to associate your own data with each connection in such a way that you never need to lock to manipulate it from connection events on the connection; you can read more about it here.

Of course the framework contains much more, see www.ServerFramework.com for full details, prices and purchasing information.. There are also several things that you can license separately to speed up your development; high performance, asynchronous SSL, SSPI ‘Negotiate’ support for Single Sign on, Kerberos and NTLM support. CLR hosting so that you can implement some of your server or client in managed code. The Performance counter library for easy server monitoring using perfmon. The Services library for easy service development. See the licensing options page of the latest framework documentation set for full details of what’s available.

Of course, you might want to ’try’ before you buy. All of the example servers are available for download as source code so that you can take a look at how real servers and clients are built using the framework; and if you want to obtain compiled versions of these examples so that you can perform your own performance tests on them then please get in touch.