Latest release of The Server Framework: 6.7

| 0 Comments
Version 6.7 of The Server Framework was released today.

This release is mainly a code simplification release which removes support for legacy compilers and operating systems. See here for more details. However, there are some breaking changes where smart buffers have replaced buffer references and this causes function signature changes. In addition there has been considerable churn in the Streaming Media Option Pack with knock on changes in the HTTP library code which needed to be made more complete to deal with serving HLS streams.

As always, see the release notes here, for full details of all changes.

Breaking changes:

  • Breaking change JetByteTools::IO::CAsyncFileReader, JetByteTools::IO:: CAsyncFileWriter, JetByteTools::IO::CAsyncFileWriterEx and JetByteTools::IO::IAsyncIOStream now work in terms of JetByteTools::IO:CSmartBuffer where possible. This potentially reduces the reference counting activity on the buffers.
  • Breaking change Where possible JetByteTools::IO::CSmartBuffer has replaced raw pointers and references to buffers. This massively reduces the need to reference count buffers during normal I/O operations and increases performance, especially on NUMA architectures.

Bug fixes:

  • Bug fixes to JetByteTools::IO::CBuffer around the usage of m_maxBytesToRead.
  • Bug fix to JetByteTools::Win32::CCallbackTimerQueueEx::BeginTimeoutHandling() to prevent incrementing m_nextTimeoutHanlde causing the value to wrap to InvalidTimeoutHandleValue which was possible, but unlikely.

Changes:
  • Dropped support for Visual Studio 2005 and Visual Studio 2008.
  • Dropped support for Windows XP.
  • Removed JETBYTE_HAS_INTERLOCKED_64 as it's true on all supported platforms now.
  • Removed JETBYTE_WARN_ON_SOCKADDR_STORAGE_DEF and JETBYTE_WARN_ON_WSPIAPI_COUNTOF_DEF as these are no longer relevant.
  • Removed JETBYTE_PERF_STREAM_SOCKETS_SKIP_EVENT_ON_HANDLE, JETBYTE_PERF_DATAGRAM_SOCKETS_SKIP_EVENT_ON_HANDLE and JETBYTE_PERF_FILE_WRITER_SKIP_EVENT_ON_HANDLE as these are always enabled now.
  • Removed JETBYTE_PERF_STREAM_SOCKETS_SKIP_MARSHAL_TO_IO_POOL and JETBYTE_PERF_DATAGRAM_SOCKETS_SKIP_MARSHAL_TO_IO_POOL as these are always enabled now. Marshalling was only required to work around Windows XP's I/O cancellation on thread termination policy.
  • Removed JETBYTE_DEPRECATE_SHARED_CRITICAL_SECTIONS, JETBYTE_DEPRECATE_SHARED_LOCK_SOCKETS, JETBYTE_DEPRECATE_CRITICAL_SECTION_2, JETBYTE_DEPRECATE_LOW_CONTENTION_BUFFER_ALLOCATOR, and JETBYTE_DEPRECATE_TLS_BUFFER_ALLOCATOR as these features are no longer available.
  • Removed SecureCRT.h as it's no longer required. It was used to map differences between the Visual Studio 2005 CRT and the "new" secure CRT.
  • Added the macro, SuppressLNK4221Warning(), which can be put into a file in order suppress the MS Visual C++ Linker warning 4221 - "warning LNK4221: no public symbols found; archive member will be inaccessible"
  • Turn off JETBYTE_INTRUSIVE_RED_BLACK_TREE_INTERNAL_STATE_FAILURE_EXCEPTIONS by default.
  • Removed JetByteTools::IO::CLowContentionBufferAllocator and JetByteTools::IO::CTLSBufferAllocator.
  • Added JetByteTools::IO::CBufferBasedBufferAlloctor which is a simple shim to allow an instance of JetByteTools::IO::IBuffer to be used as an implementation of JetByteTools::IO::IAllocateBuffers.
  • Removed JetByteTools::IO::IIOPool::DispatchToAll() as it was only required for issuing CancelIO() calls to all I/O threads on XP where CancelIOEx() wasn't available.
  • Added an overload of JetByteTools::IO::CNonPooledBuffer::Create() that takes a bufferSize and a pointer to data and a data length so that you can create a buffer that initially contains some data but that is larger than that data.
  • Added lots of standard buffer functionality to JetByteTools::IO::CNonPooledBuffer before deprecating it in favour of normal buffers obtained via JetByteTools::IO::IAllocateBuffers::AllocateCustomSizedBuffer().
  • Removed JetByteTools::IO::CAsyncFileWriter::ExecuteWritesOnCallingThreadIfSafe as it's the same as JetByteTools::IO::CAsyncFileWriter::ExecuteWritesOnCallingThread now that we no longer support Windows XP.
  • Added JetByteTools::IO::IBuffer::GetTotalLength() which returns the length of a set of buffers defined using an array of WSABUF structures.
  • Added JetByteTools::IO::CBufferChain::CopyBufferChain() which uses an instance of JetByteTools::IO::CBufferChain::IAllocateBufferHandles to create a duplicate of a given buffer chain where the duplicate contains handles to the buffers in the original chain.
  • Removed all of the code that was required to marshall I/O operations to the I/O threads on Windows XP.
  • Removed the concept of write sequencing. There's no need now that the marshalling code has been removed.
  • Removed the concept of "shared lock sockets" and, correspondingly "unique lock sockets". All socket objects now have their own lock.
  • Removed the option of setting a socket's lock's spinCount in the constructor of the socket allocator
  • Replaced some usages of JetByteTools::Win32::CCriticalSection with JetByteTools::Win32::CLockableObject
  • Removed JetByteTools::Win32::ICriticalSectionFactory, JetByteTools::Win32::ISharedCriticalSection, JetByteTools::Win32::IManageSharedCriticalSections, JetByteTools::Win32::CCriticalSection2, JetByteTools::Win32::CSharedCriticalSection, JetByteTools::Win32::CSharedCriticalSectionFactory, JetByteTools::Win32::CSmartSharedCriticalSection and JetByteTools::Win32::CUniqueCriticalSectionFactory. The concept of shared critical sections is no longer supported.
  • Removed JetByteTools::Win32::CThreadedCallbackTimerQueue::HybridTickCount64 and JetByteTools::Win32::CThreadedCallbackTimerQueue::HybridTickCount64NoLock as the hybrid GetTickCount64() implementation is no longer required as all supported platforms now provide GetTickCount64.
  • Removed JetByteTools::Win32::CCallbackTimerQueue. JetByteTools::Win32::CCallbackTimerQueueEx is now the only timer queue implementation.
  • Removed JETBYTE_USE_CAPTURE_STACK_BACK_TRACE we now ALWAYS used CaptureStackBackTrace() so there's no need to make it optional.
  • Added new overloads for JetByteTools::Win32::GetFileNameFromPathName() and JetByteTools::Win32::StripFileNameFromPathName() which takes the path separator. This allows the functions to be used for file system paths or URI paths.
  • Added new overloads for JetByteTools::Win32::GetFileVersion() and JetByteTools::Win32::GetFileVersionString() which take languge IDs and charset IDs
  • Added JetByteTools::Win32::RemoveDirectoryContents().
  • Added JetByteTools::Win32::GetFileSize().
  • Added some code to the top and bottom of Utils.h which try to deal with situations where min and max have been defined as macros. We use the std::min() and std::max() template versions and macros confuse matters so the new code attempts to undef the macros if present and then redefine them at the end of the header.

Newly deprecated code:
  • Deprecated JetByteTools::IO::IAllocateMultiBufferHandles and the concept of "multi buffer handles".
  • The stacking of connection filters that can generate their own writes has been deprecated.
  • Compressing and deflating socket filters are now deprecated.

So, it's nearly a year since I first started noticing issues with VS2015 on my build servers. The bug in question now has an entry on Microsoft Connect and Google can help with some work arounds which don't require turning the machine off and on again a random number of times until it works... There's even a Visual Studio extension that fixes the issue for you (!).

I find it disappointing that this hasn't been fixed, it's a fundamental usability issue which seems to be causing lots of people lots of pain. It's probably not too bad if you're running Visual Studio as a developer in the 'normal' way; especially if the extension can fix the issue for you when it happens, but on a build machine it's a pain. Of course, it only ever happens just after you kick off a build and leave the office. If you're sitting there waiting for it to happen the problem never seems to manifest...

I hinted at the end of the last post that the 6.7 release might increase performance a little. Well, whilst the bulk of the changes in 6.7 are purely code cleaning and the removal of legacy support there is a fairly major functional change as well.

In most situations references or pointers to I/O buffers have been replaced with smart pointers. This change may cause some issues during an upgrade as you need to change some function signatures from IBuffer refs to CSmartBuffers. The advantage is that in many servers there will no longer be any need for buffer reference counting during normal I/O operations.

The Server Framework relies on reference counting to keep the objects that are used during the asynchronous operations alive until those operations complete. So we increment a counter on the socket object and also on the buffer object when we initiate an operation and then decrement the counters when the operation completes. I'm sure there are other ways to manage the object lifetime but this has worked well for us.

The problem is that these increments, although they look like cheap operations, can be quite expensive, especially on NUMA hardware.

Whilst there's not much we can do about the reference count on the socket object, the buffer doesn't really need to be reference counted most of the time. Or, more's the point. The initial reference can (and should) be passed along rather than each stage taking and releasing its own reference. With a buffer you generally only want to be accessing it from one thread at a time and so you allocate it and then issue an operation and pass the reference you have off to the operation. When the operation completes the code captures the reference and takes ownership of it and then the buffer can be processed. If you're lucky you can then use the same buffer for a response to the operation and pass it back to the framework again.

This requires a few changes to your code but it's fairly straight forward. Your OnReadCompleted() handler will give you a CSmartBuffer and if you want to retain ownership of it after the handler returns then you simply need to detach the buffer from the CSmartBuffer you were given.

This is only "potentially faster" as it really depends on the structure of your server and how you deal with our I/O buffers but the framework is no longer standing in the way of this kind of optimisation, and we've removed a couple of reference adjustments in the normal operation flow.

Another release is coming...

| 0 Comments

We've only just shipped Release 6.6.5 of The Server Framework but we already have another release that's just about to ship. This isn't because some horrible bug has slipped through our testing, it's because we've been planning to produce a 'clean up' release for some time. 6.7 is that release.

Lets be straight here, 6.7 is a release for us more than for you. The aim is to simplify our build/test and release process, remove dead code whilst introducing no new bugs and removing no functionality that you rely on.

So what does 6.7 achieve. Well, for a start we drop support for Visual Studio 2005 and 2008 and also for Windows XP. Removing support for these legacy compilers and operating systems means that we can remove all the code that was required just to support them. This massively simplifies our code base without removing anything that the code actually relies on to run on modern operating systems.

Windows Vista introduced massively important changes to asynchronous I/O and we have supported these changes for a long time (over 8 years!). The code required to jump through hoops to make code running on Windows XP behave was complex. For example, Windows XP would cancel outstanding I/O requests if the thread that issued them exited before the I/O request completed. We had a marshalling system in place to ensure that I/O operations were only ever executed on threads that we controlled so that you'd never be faced with unexpectedly cancelled operations. All of that can go now.

Removing XP also means we no longer need to maintain an XP machine in our build farm. It's one less configuration that needs to be built and tested before a release.

Dropping support for VS2005 and 2008 removes 4 complete sets of builds (x86 and x64 for each compiler) plus all of the conditional code that was required to support the older compilers. At last we can start moving towards a slightly more modern C++, perhaps.

Some old code has been removed; there's no need, on modern operating systems, to share locks. This worked really well back in the day, but, well, we were running on Windows NT at the time and resources were much more limited than they are now. All of the "Shared Critical Section" code is now gone. This has knock on effects into the Socket Tools library where all of the shared lock socket code has been removed. Nobody should be using that in 2016 anyway! You can no longer set a critical section's spin count in the socket allocator, it never really worked anyway as the lock was used for too many different things.

Some experimental code has also been removed; The TLS and Low Contention buffer allocators are gone. The horrible "dispatch to all threads" cludge has been removed from the I/O pools (it was only there to support pre-Vista CancelIO() calls which are no longer needed now that we have CancelIOEx()).

The original callback timer queue that was based on GetTickCount() and which spawned Len's "Practical testing" series of blog posts (back in 2004!) has gone. There's no need for the complexity when all supported operating systems have GetTickCount64().

Finally we've slimmed down our set of example servers. Removing servers which didn't add much value or which duplicated other examples. Again, this speeds our release process by speeding up the build and test stage as there are fewer servers to build and fewer tests to run.

So, what's in it for you? Well, a faster build/test/release cycle so new functionality and bug fixes can be released quicker and potentially faster code in some circumstances. There's no great rush to upgrade if you don't want to, but we'll be focusing on the 6.7 code base going forwards.

Version 6.6.5 of The Server Framework was released today.

This release is mainly a feature release with a few bug fixes.

As always, see the release notes here, for full details of all changes.

Bug fixes:

  • Bug fix to JetByteTools::Socket::TAsyncSocket::ProcessAndGetNextOperation(). We now wrap the body of the function in an exception handler and abort the connection with JetByteTools::Socket::ConnectionClosureReason::FatalErrorAbort if an exception is thrown during processing. This fixes a bug whereby the connection would otherwise hang in these situations.
  • Bug fix to JetByteTools::Win32::TReentrantLockableObjectTracksLockingThread to add a cast which is needed for some compilers.
  • Bug fix to JetByteTools::WebSocket::HyBi::CProtocolHandler::HandleData() to remove an incorrect internal state validation exception which would generate spurious "No space left in read buffer." exceptions.

Changes:

  • Added JetByteTools::IO::IBuffer::OnBufferAddedToPool(), JetByteTools::IO::IBuffer::OnBufferRemovedFromPool(), JetByteTools::IO::IManageBufferLifeCycle::OnBufferAddedToPool() and JetByteTools::IO::IManageBufferLifeCycle::OnBufferRemovedFromPool(). These allow for correct management of dynamically allocated buffer data. Previously JetByteTools::IO::IMonitorBufferAllocation::OnBufferAllocated() was called whenever a buffer was allocated from the allocator OR a custom pool and JetByteTools::IO::IMonitorBufferAllocation::OnBufferReleased() was only called when the buffer was released to the allocator. This made it impossible to manage dynamically allocated buffer data that was created and destroyed by a derived allocator object using the monitoring interface to monitor the underlying allocator. Now JetByteTools::IO::IMonitorBufferAllocation::OnBufferReleased() is called when the buffer is added to a custom pool and this allows the monitor to match allocations and releases exactly.
  • Added JetByteTools::IO::IAllocateBuffers::AllocateCustomSizedBuffer() which allows you to allocate a custom buffer from any allocator. The buffer will have the same user data slots as any other buffer allocated from the allocator but if it is larger than the allocator's buffer size the new buffer will NOT be pooled upon release.
  • Added JETBYTE_ILLEGAL_BUFFER_USER_DATA_EXCEPTIONS which defaults to 0 and when set turns on range checking for user data indices in JetByteTools::IO::CBuffer and an index out of range exception is thrown if necessary.
  • Added the ability to pass JUST a custom allocator into the constructor of JetByteTools::IO::CRotatingAsyncFileLog and allow the file log to manage the other resources that it needs directly. Previously you had to provide all of the resources or none.
  • Added an override for JetByteTools::Socket::ISocketCallback::OnError() which takes a DWORD error code so that client code can selectively ignore errors by error code. Previously the only way to ignore errors was by the error message itself which is localised and therefore impossible to match reliably.
  • Rationalised the status changes for JetByteTools::Socket::TStreamSocketServer<> and JetByteTools::Socket::TStreamSocketServerEx<> to remove some strangeness when shutdowns are initiated after the server has already shut down.
  • Added new value to JetByteTools::Socket::ConnectionClosureReason, FatalErrorAbort. This is used if the framework itself needs to abort a connection for any reason.
  • Added JetByteTools::Socket::CStreamSocketNamedConnectionCollection::GetConnectionName() which returns the name of a given connection.
  • Added JetByteTools::Socket::CStreamSocketBroadcastableConnectionCollection::BroadcastToAllExcept() which broadcasts a buffer to all connections except the supplied connection.
  • Added JetByteTools::Socket::StreamSocketBroadcastableNamedConnectionCollection.
  • Added JETBYTE_TRACK_ADDRESS_REFERENCES which defaults to 0 and when set enables reference tracking of JetByteTools::Socket::CAddressImpl objects in the same way that tracking can be enabled for sockets and buffers.
  • Added JETBYTE_STREAM_SOCKETS_DISPATCH_OTHER_SOCKETS_DURING_COMPLETION_HANDLING and JETBYTE_DATAGRAM_SOCKETS_DISPATCH_OTHER_SOCKETS_DURING_COMPLETION_HANDLING which both default to 0 and this changes the default behaviour from previous releases. These control how we deal with dispatching operations from other sockets whilst dispatching events from a socket... This only affects designs where one connection can write to another connection. In such a design, in earlier versions, we would allow processing of operations from the 'other' connection whilst we're processing operations from the main connection. For example. If we're in the read completion handler for a connection and we issue a read on another connection and that read completes immediately, inline, then we would begin to handle the completion and end up in the read completion handler for the other socket. This could cause issues with lock inversions if each connection needs to take out locks. The new default is to NOT handle the inline completions for the 'other' socket inline but instead to queue them until after the current socket's operation handler completes. You can revert to the old behaviour by setting these to 1 in your Config.h file.
  • Added JETBYTE_DUMP_NAMED_INDEX_DETAILS_ON_LOCK which defaults to 0 and when set causes the names of named indices to be dumped to the default debug trace log when the indices are locked for the first time. This can be useful in tracking down mismatches between different named index providers.
  • Changed JetByteTools::Win32::ICreateMiniDumps::GenerateDumpFileName() so that it takes a 'type' which, if it's not an empty string, is added into the generated name just after the filename base portion. This allows you to group dumps by type - for situations where a you are generating dumps for different reasons.
  • Changed JetByteTools::Win32::CMiniDumper to take into account the changes to JetByteTools::Win32::ICreateMiniDumps::GenerateDumpFileName(). This means changing JetByteTools::Win32::CMiniDumper::CreateMiniDump(), JetByteTools::Win32::CMiniDumper::CreateMaxiDump() and JetByteTools::Win32::CMiniDumper::CreateFullDump() so that they take the 'type' name and adding JetByteTools::Win32::CMiniDumper::CreateMiniDumpWithFileName(), JetByteTools::Win32::CMiniDumper::CreateMaxiDumpWithFileName() and JetByteTools::Win32::CMiniDumper::CreateFullDumpWithFileName() to disambiguate certain call signatures.
  • Changed JetByteTools::Win32::CMiniDumpGenerator to take into account the changes to JetByteTools::Win32::ICreateMiniDumps::GenerateDumpFileName(). This means changing JetByteTools::Win32::CMiniDumpGeneratorGenerateDumpHere() and JetByteTools::Win32::CMiniDumpGenerator::GenerateDump() to take the new 'type' name and changing the bool used allow calls to ignore dump limits to an enum.
  • Added the concept of 'per type' mini dump limits. These can be set through JetByteTools::Win32::CMiniDumpGenerator::SetMaxDumps() by specifying a type string to set the maximums for. If a per type limit is not set then the global limit is used.
  • Added a new enum value to the JetByteTools::Win32::CMiniDumpGenerator::MaxDumpLimits enum to enable the use of 'per type' limits.
  • Changed JetByteTools::Win32::CGlobalErrorHandler to take into advantages of the dump 'type' changes to categorise the types of dumps produced.
  • Cosmetic changes to JetByteTools::Win32::CThread to adjust the optional thread name tracking structure.
  • Added JetByteTools::Win32::CPerThreadErrorLog which can be instantiated on a thread to install handlers for std::terminate and std::unexpected. These handlers are per thread. Note that you should still use JetByteTools::Win32::CGlobalErrorHandler on the main thread as this also installs some process-wide handlers.
  • JetByteTools::Win32::CGlobalErrorHandler now derives from JetByteTools::Win32::CPerThreadErrorLog.
  • The signature for the constructor of JetByteTools::Win32::CSEHException::Translator has been changed to take an int which is not used for anything except to allow you to locate 'old' instances of the class and update them to instances of JetByteTools::Win32::CPerThreadErrorLog which include the functionality of the exception translator. This change can be turned off by defining JETBYTE_BREAK_SEH_EXCEPTION_TRANSLATOR_COMPATABILITY to 0 in config.h.
  • Added JETBYTE_INSTALL_PER_THREAD_ERROR_HANDLER_IN_CTHREAD which defaults to 1 and when set installs a JetByteTools::Win32::CPerThreadErrorLog object onto all JetByteTools::Win32::CThread object's threads.
  • All threads created by framework code will install an instance of JetByteTools::Win32::CPerThreadErrorLog whether or not JETBYTE_INSTALL_PER_THREAD_ERROR_HANDLER_IN_CTHREAD is defined as 0.
  • Mini dumps created by JetByteTools::Win32::CCrtReportHook are now always created even if any program limits on the number of dumps to create has been met or exceeded.
  • Added JetByteTools::Win32::IIOCPWorkerThreadCallback::ProcessEx() which can be overridden to do the work that JetByteTools::Win32::IIOCPWorkerThreadCallback::Process() would do and then return a bool to indicate if you need to be informed when the queue has no more items to process. The default implementation simply calls JetByteTools::Win32::IIOCPWorkerThreadCallback::Process() and returns false.
  • Added JetByteTools::Win32::IIOCPWorkerThreadCallback::NoItemsToProcess() which is called once when the queue becomes empty if the last call to JetByteTools::Win32::IIOCPWorkerThreadCallback::ProcessEx() or JetByteTools::Win32::IIOCPWorkerThreadCallback::NoItemsToProcess() returned true. The default implementation simply returns false. Note that returning true will cause the queue to be checked again and if still empty for JetByteTools::Win32::IIOCPWorkerThreadCallback::NoItemsToProcess() to be called again.
  • Adjusted how services shut down in error situations when the service has failure actions set. When failure actions are enabled for a service we will now shut down in such a way that they are triggered. This involves terminating the thread that called StartServiceCtrlDispatcher() and making sure that we don't call SetServiceStatus() with a status of SERVICE_STOPPED. We shut down normally if the service is in the process or starting up or shutting down when the error occurs so that we avoid the SCM repeatedly restarting a service that we can't shut down. If "Enable actions for stops with errors" is set then the SCM will restart the service if it returns a non-zero exit code, even if it calls SetServiceStatus() with a status of SERVICE_STOPPED and there's no way for us to prevent it restarting a service that is failing during init or shutdown.
  • JetByteTools::WebSocket::CHeaders can now handle adding headers with no value.

Things have been busy!

We've been busy with lots of custom development work over the last few months. Working with Eonic Gaming on their server for the Turf Battles Triumphus 3D MMORPG. Working with one of our security company clients on their video streaming server, adding RTP over TCP and HLS streams to the RTP over UDP of the original design. Improving the performance and functionality of the pluggable server that we wrote for one of our other gaming clients as they scale out into the cloud. And doing some consultancy for existing clients who want to expand their understanding and usage of The Server Framework.

The video streaming work was fun and went well and it was nice to have a well specified project using internet standards that we could really get our teeth into.

The work for Eonic Gaming has been especially interesting as we've worked closely with them to address their initial scalability issues and switch from a single server solution to a more traditional MMORPG style multiple server design.

We've also been working on several new releases of The Server Framework, one in the 6.6.x tree, a new, slimmed down 6.7 and some investigation into what could become massively modernised 7.0 release. But we'll see...

Now that things are finally becoming a little bit calmer our focus is on getting the 6.6.5 and 6.7 releases finalised and out the door.

A while back a client of mine had an issue with a TLS 1.2 connection failing during the TLS handshake. We couldn't see any issues with the code and if we only enabled TLS 1.1 on the server then the connection handshake worked just fine.

Eventually we tracked the issue down to the fact that the certificate in use had been signed with MD5 and that MD5 isn't a valid hash algorithm for TLS 1.2 and so the handshake failed when SChannel attempted to validate the certificate and found that it was unacceptable. There's a Microsoft blog posting about this problem here.

Annoyingly the error message for this is: "The client and server cannot communicate, because they do not possess a common algorithm." which is bang on the money, but doesn't help much until after you've worked out what the problem is. We were already dumping out the enabled algorithms and protocols and couldn't understand why the connection either wasn't working or wasn't downgrading to TLS 1.1.

I've now added some certificate investigation code to the point where my example SChannel servers set up their contexts. This at least allows me to warn and degrade to TLS 1.1 if the certificate is not going to work with TLS 1.2. In production systems I expect we'd just fail to start up.

   CCredentials::Data data;

   data.cCreds = 1;
   data.paCred = &pCertContext;

   // Note that for TLS 1.2 you need a certificate that is signed with SHA1 and NOT MD5.
   // If you have an MD5 certificate then TLS 1.2 will not connect and will NOT downgrade
   // to TLS 1.1 or something else...

   DWORD dataSize = 0;

   if (::CertGetCertificateContextProperty(
      pCertContext,
      CERT_SIGN_HASH_CNG_ALG_PROP_ID,
      0,
      &dataSize))
   {
      ByteBuffer buffer(dataSize);

      BYTE *pData = buffer.GetBuffer();

      if (::CertGetCertificateContextProperty(
         pCertContext,
         CERT_SIGN_HASH_CNG_ALG_PROP_ID,
         pData,
         &dataSize))
      {
         const _tstring signHashDetails(reinterpret_cast<tchar *>(pData), dataSize);

         if (signHashDetails.find(_T("MD5")) != _tstring::npos)
         {
            if (enabledProtocols == 0 || enabledProtocols & SP_PROT_TLS1_2)
            {
               OutputEx(_T("Certificate is signed with: ") + signHashDetails);
               OutputEx(_T("MD5 hashes do not work with TLS 1.2 and connection downgrade does NOT occur"));
               OutputEx(_T("This results in failure to connect. Disabling TLS 1.2"));

               if (enabledProtocols)
               {
                  enabledProtocols &= ~SP_PROT_TLS1_2;
               }

               if (!enabledProtocols)
               {
                  // if enabledProtocols is zero then all protocols are
                  // enabled, so we need to explicitly enable everything
                  // except TLS1.2

                  enabledProtocols = SP_PROT_SSL3TLS1;
              }
            }
         }
      }
   }

   data.grbitEnabledProtocols = enabledProtocols;

Latest release of The Server Framework: 6.6.4

| 0 Comments
Version 6.6.4 of The Server Framework was released today.

This release is mainly a bug fix release for clients using WebSockets over SSL.

As always, see the release notes here, for full details of all changes.

Bug fixes:

  • Bug fix to JetByteTools::Win32::CallbackTimerQueueBase which prevents the timeout handle from ever being incremented to zero. It's unlikely but possible.
  • Bug fix to JetByteTools::Win32::CBuffer and JetByteTools::Win32::CLockFreeBuffer to make code take notice of when m_maxBytesToRead is set. This was causing issues with WebSockets over SSL where the WebSocket code was setting a max read value and the SSL connector was ignoring it and providing more data than it should.
  • Bug fix to JetByteTools::SSPI::SChannel::CAsyncConnector::PerformWrite() so that we deal with exceptions appropriately.
  • Bug fix to JetByteTools::Socket::CFilterDataBase::WriteData() so that we deal with exceptions appropriately.
  • Bug fix for a highly unlikely possible situation in JetByteTools::WebSocket::HyBi::CProtocolHandler::HandleData() where a buffer in the read request buffers queue contains no space for more data.

Changes:

  • Adjusted all includes to remove relative paths and instead be absolute in relation to the library root. So, rather than #include "..\Blah.h" it would always be #include "JetByteTools\Lib\Blah.h" this is due to a new warning in Visual Studio 2015 update 1.
  • Added JetByteTools::Socket::ListenBacklogMaxwhich is a JetByteTools::Socket::ListenBacklog value that can be used to cause the listen queue for a socket to be set to SOMAXCONN.
  • Added JetByteTools::Socket::TStreamSocketServerEx::SoMaxNumPendingAccepts which is the number of accepts that are posted if a listen backlog of JetByteTools::Socket::ListenBacklogMax is specified for the number of pending accepts. At present this is set to 200.
  • Added JetByteTools::Socket::TStreamSocketServerEx::SetListeningParameters() which can be called after construction but before starting to accept connections and can specify the listen backlog and number of receives to post separately. This allows you to set the listen backlog to SOMAXCONN using JetByteTools::Socket::ListenBacklogMax and set the number of receives to something other than JetByteTools::Socket::TStreamSocketServerEx::SoMaxNumPendingAccepts. You can also set a flag which tells the server not to close the listening socket when the server reaches a connection limit, instead it will simply stop posting accepts and so new connections will get to wait in the listen queue until they time out or until the server posts a new accept request.
  • Bug fix to to JetByteTools::Socket::TStreamSocketServerEx so that OnMaxConnections() is called correctly when the connection limit is reached. Previously it would often only be called the first time that the limit was reached.

SChannel ALPN support, documentation "bug"...

| 0 Comments

I'm looking at adding support for the TLS Application-Layer Protocol Negotiation Extension to The Server Framework as a precursor to looking into HTTP 2.0 support. This TLS extension is supported by both SChannel and OpenSSL. It's relatively new (only having been included in SChannel since June 2014 (!)) but there seems to be no documentation on MSDN for the structures and how to use them to add the appropriate entries to the TLS handshake and then to select and validate the protocol of your choice... Searching today with Google for something like "SEC_APPLICATION_PROTOCOL_NEGOTIATION_EXT" just gives just the usual selection of hits to people who think it's clever to run the Windows header files through their document generators and then publish the results on the web...

It's slightly easier with OpenSSL, but only because I can look at the source...

Anyway, long story short, I have the OpenSSL version working and am getting an HTTP 2.0 Connection Prefix from Chrome... I guess, for now, SChannel will have to wait.

I expect I'm being dense...

There should be a Developer Advocate position within Microsoft. They seriously need someone who doesn't "Drink the Kool-Aid" who can step in and tell the Program Managers or Marketing bods or whoever it is who makes all the really STUPID decisions that the decision is stupid and that they should get back in their box and leave the developers who support the Windows ecosystem by building software that runs on it to just get on with their lives...

Yes, this is going to be a rant...

So someone has decided that being able to query which version of the Operating System that your program is running on is a bad thing. If you try and call GetWindowsVersionEx() on Windows 8.1 or Windows 10 and you haven't specifically set your application's manifest to say that it's compatible with Windows 8.1 or Windows 10 then GetWindowsVersionEx() will lie to you and tell you that you're running on Windows 8.

Excuse me, but that's just brain dead. There's NO valid reason for doing this because you can't know WHY the caller needs this information.

OK, so it might be better, in some circumstances, to be checking for the presence of various function entry points in various dlls and dynamically working out that what you need is present BUT that's not the only reason that you might want to know what OS you're running on and it's not even the only GOOD reason.

My current situation is that Windows 10 spawns a conhost.exe process whenever you start a console process. The test code that I have for my Win32 Job API wrapper includes some checks to determine that the correct job accounting data is being returned from some test programs that it spawns. This accounting data is different under Windows 10 as there are more processes running within the job. Ideally I'd like to acknowledge that this is an OS difference, adjust the test to know that when it runs on this OS it needs to expect slightly different results and then move on with my life. Of course, as Microsoft has deliberately chosen to break the API that would let me cleanly work out what version I'm running on I have to spend time writing ranty blog postings and crufting up crappy work arounds.

About this Blog

I usually write about C++ development on Windows platforms, but I often ramble on about other less technical stuff...

This page contains recent content. Look in the archives to find all content.

I have other blogs...

Subscribe to feed The Server Framework - high performance server development
Subscribe to feed Lock Explorer - deadlock detection and multi-threaded performance tools
Subscribe to feed l'Hexapod - embedded electronics and robotics
Subscribe to feed MegèveSki - skiing