Latest release of socket server code

| 57 Comments

The latest release of the free version of my asynchronous, windows, IOCP based, socket server framework can now be obtained from here at ServerFramework.com.

The latest release is really just a bug fix and compiler compatability release. The code now supports Visual Studio 2005 (as well as VC6, Visual Studio .Net (2002) and Visual Studio 2003). I've fixed the memory leak bugs that were present in the ThreadPoolLargePacketEchoServer sample and worked around the iostream leak that's present in the Visual Studio 2005 version of STL. I've also included a set of build configurations for the basic echo server that builds the system with STLPort rather than the standard Visual Studio STL.

The bug fixes are reported here. Please note that this version of the code still suffers from poor handling of low resource issues; see here for the kind of things that could leave you with an unresponsive server.

I expect that this will be the last release of the free code. Though, I expected that last time too ;)

The example servers are as follows:


  • EchoServer - the basic example server that simply echoes back the data it receives.

  • PacketEchoServer - a server that echoes back complete packets, each packet starts with a length indicator. This is slightly more "real world" as many protocols have a header that indicates the length of the packet that follows.

  • SimpleProtocolServer - a server that operates on variable length packets that are terminated with a \r\n sequence. This is the other common way to determine how much data the client is sending in one "command". Things like SMTP and POP3 use this style of protocol.

  • SimpleProtocolServer2 - a version of the SimpleProtocolServer above that uses a separate thread pool to actually process the requests. This is potentially useful if the requests will block (such as requests to a database, perhaps). If you have X IO threads and all of them are engaged in a blocking activity at once then your communications subsystem stops responding! This server architecture avoids that problem by passing complete command packets to another thread pool via an in process FIFO queue so that the blocking operations do not adversly affect the communications subsystem.

  • ThreadPoolLargePacketEchoServer - a really contrived example that I've only ever used to demonstrate some theory. This server is a packet echo server that runs the communications subsystem with a small buffer size and yet requires that the server only echo packets that do not fit completely in the small buffer size. This means that the server copies data from the IO buffers into "transfer buffers" that it then passes to another thread pool to echo. I've never found a use for this architecture but it might be useful if your protocol uses buffers that are 99% one size and occasionally bigger...

I recommend using the simplest server architecture that you can. That means you're probably looking for something based on either the second or third example. If you access a database or other slow, blocking, non async device then you should probably use a variation on server four. If you think you really need to use the fifth server as a base get in touch and I'll talk you out of it ;)

Due to how I have my CVS repository structured and how I like to work on servers in isolation (I sometimes work on multiple servers at once and like to build/rebuild them without affecting other work in progress), each server has its own copy of the "tools" library.

Please report any bugs that you may find via email.

You probably need the platform SDK to build the code. See disclaimers on the previous release and on the original code project articles for more details.

If you need more than this free code provides then take a look at the rest of ServerFramework.com for details of The Server Framework a licensed and fully supported high performance server framework.

Updated: 18 Feb - fixed a bug in the new version of ToString()

Updated: 25 Feb - added the files for the CStringConverter class so that the fix I added on the 18th actually compiles!

Updated: 27 Oct - bug fix for CSocket::InternetAddress - see here for details

Updated: 14 March - link to the differences between this free version of the code and the licensed version: see here.

Updated: 10 Jan 2008 - bug fix for the write sequencing memory leak and data loss - see here for details

Updated: 12 Jan 2008 - bug fix for the potential deadlock bug - see here for details

Updated: 21 April 2009 - see here for a bug fix for a connection termination issue that leaves I/O threads unresponsive (note that this fix is not currently included in the code on this page.

Updated: 4 October 2010 - moved the code to ServerFramework.com

57 Comments

There seems to be some wierdness with the new implementation of ToString() on VC6. Will post a fix when I have one.

The wierdness was me ;) The new version of ToString() used std::wostringstream for the wide character version whereas the original code used the narrow version and then converted the result. There's a good reason for doing it the old way... Since on VC6 (and optionally on later compilers) wchar_t is, in fact, unsigned short the wide streams can't stream an unsigned short as they assume it's a string...

Hi Len,

I was testing various completion port based server solutions and finally I decided to play longer with your solution. I am really impressed with the quality and usability of your JetByte server tools.

I found a small problem trying to use "SocketServers-LatestRelease.zip". The compiler complains that it cannot find "CStringConverter::AtoW" when building unicode version of SimpleProtocolServer2. Is the CStringConverter class a standard one?

Thanks for the compliment.

I'll check the build, I adjusted a couple of includes at the last minute due to the ToString() changes and I expect it's a precompiled header issue.

Trianon,

I've fixed that problem and uploaded a new zip file. There were two missing files (for the CStringConverter class). This came about because I first fixed the VS2005 ToString() leak in this code and then ported it to my latest libraries, they don't use USES_CONVERSION as it's evil ;). I added some tests to the new code and found the unsigned short issue with the new code so I fixed the bug in the new lib and then ported the code back to the old socket server codebase. Looks like I didn't try a compile before I shipped it! Unusual for me to be this slack but I've had a bit of a week.

Hi Len,

thanks for the wonderful code. I'm (re)using your framework and it works great. One thing that i'm trying to achieve is to introduce priority queues in the framework so I can provide a prioritized service depending on a type of the message. Would you have any suggestion what approach to take. I would really appreciate your thoughts.

best regrads.

I dont have a priority queue class. I guess I'd probably build one using an std::list of std::lists of items a critical section to make it thread safe and a semaphore to allow me to wait on it having entries...

Once you have that working then you could replace the IOCP between the IO threads and the "business logic" threads with your priority queue and you're done...

Hi Len,
This library looks terrific. You mention that we can pay a fee to subscribe to get the latest version. What is the fee and how do we subscribe? I don't see your e-mail address on here anywhere.

David

David,

Contact details are on the main page, on the right, under links...

I've emailed you details.

First of all, your great solution.
In this version, When the client's socket is disconnected,
CThreadPoolWorkerThread::OnConnectionClosing function is called. But sometimes is not called.
Is something wrong?
Also CThreadPoolWorkerThread::OnConnectionClosed function is not implemented.
If I want to call OnConnectionClosed, how do I?

For example, I customized the code like below.
Is right? :(

enum DispatchEvents
{
ConnectionEstablished,
ConnectionClosing,
ConnectionClosed, // added
ReadCompleted
};

bool CSocketServer::OnConnectionClosed( Socket *pSocket)
{

Output(_T("OnConnectionClosing"));
m_pool.DispatchConnectionClosed(pSocket);
return true;
}

// added new func
void CThreadPool::DispatchConnectionClosed(
Socket *pSocket)
{
DoDispatch(pSocket, 0, ConnectionClosed);
}

sorry for my english.

OnConnectionClosing() is called if the server initiates the socket closure, you should also handle OnConnectionClientClose() to deal with situations where the client closes the connection and OnConnectionReset() for errors, etc.

OnConnectionClosed() is called once the socket itself has been closed, this may be in response to an explicit call to Close() from the server code or due to the reference count on the socket reaching 0.

Hi Len
I would like to implement a server containing both the broadcast server and the tcpserver handling multiple connections, do I need to create one iocp handle for both or just use one iocp handle for each?

I dont understand the question... Are you using my source code? If so I assume you're asking if you can create one instance of the IOPool and share that between servers? If so, then yes, see the EchoServer example code for details.

Hi Len,

Suppose I have to connect() to some other tcp based servers in parallel with running your tcp server. Can i connect() to the any remote server and add my socket to the iocp handle to participate in iocp events for WSARecv() and WSASend() completion events for those sockets? If yes, how to do this?

Thanks.

Yes, all of the new servers allow outbound connections and they share the iocp, io threads, socket and buffer pools with any inbound connections. You can just call connect() (sync or async) on the server and you'll get a socket that's connected to a remote end point.

See here for the docs for the connection manager (which can be used alone if all you need are outbound connections) which is the base class of the new server class.

Hi,

Thanks for the reply.

I am wondering, where are the docs? :)

Link was broken, now fixed.

Thank you for fixing the link.
Is this doc for the licenced or the free version of the sockets server? If it is for the free version, how can I have a look at the source code?

It's for the licensed version. There are no docs for the free version except all of the blog postings and codeproject articles. The latest free version does support sync connect via the Connect() call on the server object. The code for the free version can be downloaded from this page (near the bottom of the article body.).

Hi Len,

Nomrally, when writing data to clients connected to CSocketServer, WriteCompleted() callback is implemented that will write data to the socket passed in. If the data that is to be written to the socket has larger buffer size than the default IO buffer size defined when instantiating the CSocketServer derived class, the code is simply failing... the IOBuffer class is not chunking the data passed in using AddData().

I noticed that you are using buffer sequencing. Is that effectively working? How can i utilize it? Do i need to chunk my own data before passing it into CIOBuffer::AddData() and WSASend() takes place? If yes, is sequencing taking effect here?

Please tell me if I am missing anything here or the server code needs to be extended for this to work properly.

Thanks.

I believe I have to keep track of the internal buffer size and not to write data larger than the CIOBuffer size each time Write completes and a new write is to take place. Right?

That was my conclusion after looking at the code for SimpleSocketServer2 example.

I believe also that CIOBuffer should take care of this chunking instead of per connection details. Right?

The chunking should be done by the call to Write, on the socket, with a const char * and data length. It doesnt seem to be in the version of the code that's here, i'll try and dig out a version of the code that does it.

Oh, and it's not the buffer's business to care about what you do with it, the socket write function knows how to allocate buffers and split the data.

Hi Len,

Are you going to post a version that does the chuncking on the socket with a const char* and a data length?

In the write, in SocketServer.cpp that starts like this:


bool CSocketServer::Socket::Write(
const BYTE *pData,
size_t dataLength,
bool throwOnFailure /* = false*/)
{

Replace this:
   CIOBuffer *pBuffer = m_server.Allocate();

pBuffer->AddData(pData, dataLength);

pBuffer->SetSequenceNumber(GetSequenceNumber(WriteSequenceNo));

m_server.PostIoOperation(this, pBuffer, IO_Write_Request);

pBuffer->Release();

return true;
}

With something like this:
   size_t remaining = dataLength;

size_t index = 0;

do
{
CIOBuffer *pBuffer = m_server.Allocate();

const size_t sent = min(remaining, pBuffer->GetSize());

pBuffer->AddData(pData + index, sent);

index += sent;
remaining -= sent;

pBuffer->SetSequenceNumber(GetSequenceNumber(WriteSequenceNo));

m_server.PostIoOperation(this, pBuffer, IO_Write_Request);

pBuffer->Release();
}
while (remaining != 0);

return true;
}

Note that this hasn't been tested, or even compiled.

Note that you MUST be using sequencing for this to send the data in the right order...

Using sequeucing means setting the useSequenceNumbers variable to true in the call to CSocketServer ctor (which is by default equals true). right?

I have a question if possible.

If a WSASend() call did not send all buffer passed in, how will the server detect this and act accordingly? Which piece of code handles this?

WriteCompleted() will have a buffer supplied to it in which pBuffer->GetUsed() != pBuffer->GetWSABUF()->len. There's a test for this in the default handler and an error message is produced.

However... if you ever have more than one write issued then there's not a great deal you can do as subsequent writes may have completed by the time you try and send the rest of the data and therefore your data stream is corrupted....

Hi Len,

When creating outbound connections to remote server using the socket server's framework you use CSocketServer::Connect() method. How can I receive notifications that connections were established or created for outbound connections. I tried to trap WorkerThread::OnConnectionEstablished(), however, call stack did not reach it. Also, there is a call to OnConnectionCreated() when using CSocketServer::Connect() for outbound connections that is only when the free list of sockets is empty.

Could you please help me in this? Thanks a lot.

There is no callback for that in the free version of the code. It's assumed that since the only connect that's available is synchronous and that since it throws an exception when it fails it's fairly obvious to the caller when an outgoing connection has been established and they can do whatever they want there and then with the socket that is returned from the call to connect...

The OnConnectionCreated() that you refer to on the CSocketServer is part of the 'socket pool' callbacks and, as such, is only called when a new connection (socket) is created for use by the server (ie not removed from the pool) it's for monitoring the number of sockets that you have created, not the number of connections that you have.

It's a bit different in the licensed code as we support async connects and so there has to be a callback to report when they complete and so the two flavours of sync connect also support the callbacks ... And the server has been broken into logical lumps that do a single thing (such as socket allocators to allocate sockets and buffer allocators to allocate buffers), the callback interfaces are all explicit (which makes it easier to work out which methods do what) and so the monitoring is easier to see (and potentially ignore) and the actual connection oriented callbacks are easier to understand...

Sir,

I have modified CSocketServer::Connect() to look like this: (just to be able to receive notifications on connection established event)
Connection release is already there (right?)

CSocketServer::Socket *CSocketServer::Connect(const sockaddr_in &address)
{
CSocket soc(CreateOutboundSocket(m_address, 0));
soc.Connect(address);
Socket *pSocket = AllocateSocket(soc.Detatch());
CIOBuffer *pAddress = Allocate();
int addressSize = (int)pAddress->GetSize();
memcpy(const_cast(pAddress->GetBuffer()), &address, addressSize);
pAddress->Use(addressSize);
OnConnectionEstablished(pSocket, pAddress);
pAddress->Release();
return pSocket;
}

Do you think this would generate any negative side effect on outbound connected sockets? memory leaks?

Looks fine as long as your code in OnConnectionEstablished() doesnt throw exceptions.

Hi Len,

I am using your server code for SimpleTcpProtocolServer2 to build an application.

The application requires the server startup code to run from a different thread. I have created a callback timer to start the server. I am noticing that the server is starting and promptly destroying the thread pool with an error on the console saying:

~CThreadPool() - Shutting down worker threads

the server then terminates in that thread. All thread die.

What could cause such a behavior?

When running the server from the main thread, every thing goes OK. Any hints?

Sounds like your thread pool is being destroyed prematurely... If you need to run initialisation on a thread then you need to make sure that you either dynamically allocate all of the objects that you need to survive the end of the thread's lifetime or you keep the thread around to keep the stack based objects alive.

I could review the code for you if you want, let me know and I'll send you a quote by email.

Thank you sir for your appreciated help.
I have nailed down the problem, it was an unhandled exception that forced the server to terminate. My apologies!

Hello Len,

I am developing a server application using your SocketServerProtocol2 sample application. The server writes data to connected clients. Data sent to clients is not sent on a steady manner. Data is sent as the server detects new data is there.

So, most of the time, clients will not be sent
data. Only when data comes in, it will be sent.

Is there a way to detect client abrupt disconnection using your framework?

My solution for this was to have one zero bytes read operation pending for each connection. When it completes, it is been reposted again. Is that OK or do you have any other more profound suggestions?

Thank you.

Always have a read pending for each connection. When the client disconnects you get a notification. However, remember that TCP/IP is designed to handle (and hide) service interruptions. You can only be sure that the connection is broken if you try and send and the send fails (after all of the TCP stack's retries). You can pull out cables between client and server routers and (if data isn't flowing) you'll never know (or need care)...

I think, using pending reads for each connection can put out the overhead that we should maintain a socket reference if we want to keep track of idle clients (if no reads or writes are actually taking place). Right?

I found a bug in Utils.cpp. You use std::auto_ptr for arrays.

erdbeer,

Funny you should mention that, I found the same thing a couple of days ago when I was testing out Bounds Checker on the code. There'll be a new release of the free code eventually.

Could you show us the details of the issue with using std::auto_ptr arrays?

Den,

There are a few places where there's code like this:
auto_ptr<TCHAR> spBuf(new TCHAR[size]);

The issue is that auto_ptr uses delete but the new above needs delete [] to be called. This most likely results in a memory leak (only the first element is deleted). In most places the fix is probably to use std::string as a buffer, or to use the TExpandableBuffer class.

std::vector is the fix!

It's a fix, yes.

Hi Len,

I had been using your socket server framework for a while. Recently, I require to add an enhancement to the source code. I would like to keep the connected clients alive, until the client itself wants to disconnect or some network failure, or some time-out had occurred.

Please tell me if my approach is connected or not,

1) I can put an addref in your socket pointer and then store them per user ID after a new connection arrives. The list I maintaine will be in the socketserver base class and I will put a mutext protection everytime I use it.
Now I need to do socket->release on OnConnectionClientClose and OnConnectionReset.

Assuming no pending read and write, what is the clean way for the server to end these keep-alived connections that I held in the list? Is it psocket->shutdown good enough, or do I have to do socket->release in source code? From the shutdown function, I can't see how it go to onsocketrelease, at which I will clean my per socket data at that time.

2) I had used your CCallbacktimer in server source code, except it has one bug. The gettickcounts function does not work after 50 days. Can I just use a timestamp such as CTime and CTimeSpan to calculate the time difference and call the on-timer function when it was expired? Do you have a better solution?

Dennis

Hi Dennis,

1) - Yes, that's correct. You need to release the socket when you're done with it and remove it from the collection. So, in OnConnectionClientClose and OnConnectionReset to deal with the connection being closed or reset by the other side and also when you want the server to end the connection. You should call Shutdown() to initiate the connection termination and then Release() because you're now done with the socket.

2) Yes, I know about the bug. See http://www.lenholgate.com/archives/000306.html for a whole series of articles about how to fix it (and how to write tests for it whilst fixing it!). The latest articles contain fixed versions of the class.

hi

in CThread, I find you create thread by _beginthreadex(), but Terminate it by TerminateThread(), is there an issue? from SDK document, should Terminate by _endthreadex()

_endthreadex() is a function that you call FROM the thread that you want to terminate. And if you dont call it and simply return from the thread function then it's called automatically for you (if you started the thread wih _beginthreaded().

TerminateThread() allows you to terminate ANOTHER thread; that is you call if from one thread to terminate another thread.

So, in answer to your question; No. There is no problem. There is no other way to do what we do in that code.

how about your licensed version of socket server library?Such as price and...

Hi,

I am using your server classes for one of my applications. How can I get connected client's ip-address at server in function ReadCompleted()??


Thanks and Regards,
Ajay.

Leave a comment

About this Entry

VS2005 STL strstream and stringstream leaks was the previous entry in this blog.

Table for one is the next entry in this blog.

I usually write about C++ development on Windows platforms, but I often ramble on about other less technical stuff...

Find recent content on the main index or look in the archives to find all content.

I have other blogs...

Subscribe to feed The Server Framework - high performance server development
Subscribe to feed Lock Explorer - deadlock detection and multi-threaded performance tools
Subscribe to feed l'Hexapod - embedded electronics and robotics
Subscribe to feed MegèveSki - skiing