/* Rambling comments... */

There seems to be some wierdness with the new implementation of ToString() on VC6. Will post a fix when I have one.

Posted by: Len at February 15, 2006 10:17 AM

The wierdness was me ;) The new version of ToString() used std::wostringstream for the wide character version whereas the original code used the narrow version and then converted the result. There's a good reason for doing it the old way... Since on VC6 (and optionally on later compilers) wchar_t is, in fact, unsigned short the wide streams can't stream an unsigned short as they assume it's a string...

Posted by: Len Holgate at February 18, 2006 11:53 AM

Hi Len,

I was testing various completion port based server solutions and finally I decided to play longer with your solution. I am really impressed with the quality and usability of your JetByte server tools.

I found a small problem trying to use "SocketServers-LatestRelease.zip". The compiler complains that it cannot find "CStringConverter::AtoW" when building unicode version of SimpleProtocolServer2. Is the CStringConverter class a standard one?

Posted by: trianon at February 25, 2006 01:32 AM

Thanks for the compliment.

I'll check the build, I adjusted a couple of includes at the last minute due to the ToString() changes and I expect it's a precompiled header issue.

Posted by: Len at February 25, 2006 06:49 AM

Trianon,

I've fixed that problem and uploaded a new zip file. There were two missing files (for the CStringConverter class). This came about because I first fixed the VS2005 ToString() leak in this code and then ported it to my latest libraries, they don't use USES_CONVERSION as it's evil ;). I added some tests to the new code and found the unsigned short issue with the new code so I fixed the bug in the new lib and then ported the code back to the old socket server codebase. Looks like I didn't try a compile before I shipped it! Unusual for me to be this slack but I've had a bit of a week.

Posted by: Len at February 25, 2006 07:53 AM

Hi Len,

thanks for the wonderful code. I'm (re)using your framework and it works great. One thing that i'm trying to achieve is to introduce priority queues in the framework so I can provide a prioritized service depending on a type of the message. Would you have any suggestion what approach to take. I would really appreciate your thoughts.

best regrads.

Posted by: Alek at April 7, 2006 06:10 PM

I dont have a priority queue class. I guess I'd probably build one using an std::list of std::lists of items a critical section to make it thread safe and a semaphore to allow me to wait on it having entries...

Once you have that working then you could replace the IOCP between the IO threads and the "business logic" threads with your priority queue and you're done...

Posted by: Len at April 7, 2006 07:29 PM

Hi Len,
This library looks terrific. You mention that we can pay a fee to subscribe to get the latest version. What is the fee and how do we subscribe? I don't see your e-mail address on here anywhere.

David

Posted by: David at April 15, 2006 04:21 PM

David,

Contact details are on the main page, on the right, under links...

I've emailed you details.

Posted by: Len at April 15, 2006 04:38 PM

First of all, your great solution.
In this version, When the client's socket is disconnected,
CThreadPoolWorkerThread::OnConnectionClosing function is called. But sometimes is not called.
Is something wrong?
Also CThreadPoolWorkerThread::OnConnectionClosed function is not implemented.
If I want to call OnConnectionClosed, how do I?

For example, I customized the code like below.
Is right? :(

enum DispatchEvents
{
ConnectionEstablished,
ConnectionClosing,
ConnectionClosed, // added
ReadCompleted
};

bool CSocketServer::OnConnectionClosed( Socket *pSocket)
{

Output(_T("OnConnectionClosing"));
m_pool.DispatchConnectionClosed(pSocket);
return true;
}

// added new func
void CThreadPool::DispatchConnectionClosed(
Socket *pSocket)
{
DoDispatch(pSocket, 0, ConnectionClosed);
}

sorry for my english.

Posted by: null at September 19, 2006 03:22 AM

OnConnectionClosing() is called if the server initiates the socket closure, you should also handle OnConnectionClientClose() to deal with situations where the client closes the connection and OnConnectionReset() for errors, etc.

OnConnectionClosed() is called once the socket itself has been closed, this may be in response to an explicit call to Close() from the server code or due to the reference count on the socket reaching 0.

Posted by: Len at September 20, 2006 11:16 PM

Hi Len
I would like to implement a server containing both the broadcast server and the tcpserver handling multiple connections, do I need to create one iocp handle for both or just use one iocp handle for each?

Posted by: rbshen at March 6, 2007 10:03 PM

I dont understand the question... Are you using my source code? If so I assume you're asking if you can create one instance of the IOPool and share that between servers? If so, then yes, see the EchoServer example code for details.

Posted by: Len at March 7, 2007 02:56 AM

Hi Len,

Suppose I have to connect() to some other tcp based servers in parallel with running your tcp server. Can i connect() to the any remote server and add my socket to the iocp handle to participate in iocp events for WSARecv() and WSASend() completion events for those sockets? If yes, how to do this?

Thanks.

Posted by: Firas at May 10, 2007 05:25 PM

Yes, all of the new servers allow outbound connections and they share the iocp, io threads, socket and buffer pools with any inbound connections. You can just call connect() (sync or async) on the server and you'll get a socket that's connected to a remote end point.

See here for the docs for the connection manager (which can be used alone if all you need are outbound connections) which is the base class of the new server class.

Posted by: Len at May 10, 2007 05:37 PM

Hi,

Thanks for the reply.

I am wondering, where are the docs? :)

Posted by: Firas at May 10, 2007 06:10 PM

Link was broken, now fixed.

Posted by: Len at May 10, 2007 06:18 PM

Thank you for fixing the link.
Is this doc for the licenced or the free version of the sockets server? If it is for the free version, how can I have a look at the source code?

Posted by: Firas at May 10, 2007 06:22 PM

It's for the licensed version. There are no docs for the free version except all of the blog postings and codeproject articles. The latest free version does support sync connect via the Connect() call on the server object. The code for the free version can be downloaded from this page (near the bottom of the article body.).

Posted by: Len at May 10, 2007 06:28 PM

Hi Len,

Nomrally, when writing data to clients connected to CSocketServer, WriteCompleted() callback is implemented that will write data to the socket passed in. If the data that is to be written to the socket has larger buffer size than the default IO buffer size defined when instantiating the CSocketServer derived class, the code is simply failing... the IOBuffer class is not chunking the data passed in using AddData().

I noticed that you are using buffer sequencing. Is that effectively working? How can i utilize it? Do i need to chunk my own data before passing it into CIOBuffer::AddData() and WSASend() takes place? If yes, is sequencing taking effect here?

Please tell me if I am missing anything here or the server code needs to be extended for this to work properly.

Thanks.

Posted by: Firas at May 13, 2007 01:55 PM

I believe I have to keep track of the internal buffer size and not to write data larger than the CIOBuffer size each time Write completes and a new write is to take place. Right?

That was my conclusion after looking at the code for SimpleSocketServer2 example.

I believe also that CIOBuffer should take care of this chunking instead of per connection details. Right?

Posted by: Firas at May 13, 2007 04:21 PM

The chunking should be done by the call to Write, on the socket, with a const char * and data length. It doesnt seem to be in the version of the code that's here, i'll try and dig out a version of the code that does it.

Oh, and it's not the buffer's business to care about what you do with it, the socket write function knows how to allocate buffers and split the data.

Posted by: Len at May 13, 2007 06:30 PM

Hi Len,

Are you going to post a version that does the chuncking on the socket with a const char* and a data length?

Posted by: Firas at May 17, 2007 07:21 AM

In the write, in SocketServer.cpp that starts like this:

bool CSocketServer::Socket::Write(

   const BYTE *pData, 

   size_t dataLength,

   bool throwOnFailure /* = false*/)

{

Replace this:

   CIOBuffer *pBuffer = m_server.Allocate();

  

   pBuffer->AddData(pData, dataLength);

  

   pBuffer->SetSequenceNumber(GetSequenceNumber(WriteSequenceNo));

  

   m_server.PostIoOperation(this, pBuffer, IO_Write_Request);

  

   pBuffer->Release();

  

   return true;

}

With something like this:

   size_t remaining = dataLength;

  

   size_t index = 0;

  

   do

   {

      CIOBuffer *pBuffer = m_server.Allocate();

  

      const size_t sent = min(remaining, pBuffer->GetSize());

  

      pBuffer->AddData(pData + index, sent);

  

      index += sent;

      remaining -= sent;

  

      pBuffer->SetSequenceNumber(GetSequenceNumber(WriteSequenceNo));

  

      m_server.PostIoOperation(this, pBuffer, IO_Write_Request);

  

      pBuffer->Release();

   }

   while (remaining != 0);

  

   return true;

}

Note that this hasn't been tested, or even compiled.

Note that you MUST be using sequencing for this to send the data in the right order...

Posted by: Len at May 17, 2007 07:47 AM

Using sequeucing means setting the useSequenceNumbers variable to true in the call to CSocketServer ctor (which is by default equals true). right?

Posted by: Firas at May 17, 2007 07:55 AM

Correct.

Posted by: Len at May 17, 2007 07:58 AM

I have a question if possible.

If a WSASend() call did not send all buffer passed in, how will the server detect this and act accordingly? Which piece of code handles this?

Posted by: Firas at May 17, 2007 09:58 AM

WriteCompleted() will have a buffer supplied to it in which pBuffer->GetUsed() != pBuffer->GetWSABUF()->len. There's a test for this in the default handler and an error message is produced.

However... if you ever have more than one write issued then there's not a great deal you can do as subsequent writes may have completed by the time you try and send the rest of the data and therefore your data stream is corrupted....

Posted by: Len at May 17, 2007 10:06 AM

Hi Len,

When creating outbound connections to remote server using the socket server's framework you use CSocketServer::Connect() method. How can I receive notifications that connections were established or created for outbound connections. I tried to trap WorkerThread::OnConnectionEstablished(), however, call stack did not reach it. Also, there is a call to OnConnectionCreated() when using CSocketServer::Connect() for outbound connections that is only when the free list of sockets is empty.

Could you please help me in this? Thanks a lot.

Posted by: Firas at May 17, 2007 01:52 PM

There is no callback for that in the free version of the code. It's assumed that since the only connect that's available is synchronous and that since it throws an exception when it fails it's fairly obvious to the caller when an outgoing connection has been established and they can do whatever they want there and then with the socket that is returned from the call to connect...

The OnConnectionCreated() that you refer to on the CSocketServer is part of the 'socket pool' callbacks and, as such, is only called when a new connection (socket) is created for use by the server (ie not removed from the pool) it's for monitoring the number of sockets that you have created, not the number of connections that you have.

It's a bit different in the licensed code as we support async connects and so there has to be a callback to report when they complete and so the two flavours of sync connect also support the callbacks ... And the server has been broken into logical lumps that do a single thing (such as socket allocators to allocate sockets and buffer allocators to allocate buffers), the callback interfaces are all explicit (which makes it easier to work out which methods do what) and so the monitoring is easier to see (and potentially ignore) and the actual connection oriented callbacks are easier to understand...

Posted by: Len at May 17, 2007 02:03 PM

Sir,

I have modified CSocketServer::Connect() to look like this: (just to be able to receive notifications on connection established event)
Connection release is already there (right?)

CSocketServer::Socket *CSocketServer::Connect(const sockaddr_in &address)
{
CSocket soc(CreateOutboundSocket(m_address, 0));
soc.Connect(address);
Socket *pSocket = AllocateSocket(soc.Detatch());
CIOBuffer *pAddress = Allocate();
int addressSize = (int)pAddress->GetSize();
memcpy(const_cast(pAddress->GetBuffer()), &address, addressSize);
pAddress->Use(addressSize);
OnConnectionEstablished(pSocket, pAddress);
pAddress->Release();
return pSocket;
}

Do you think this would generate any negative side effect on outbound connected sockets? memory leaks?

Posted by: Firas at May 17, 2007 03:37 PM

Looks fine as long as your code in OnConnectionEstablished() doesnt throw exceptions.

Posted by: Len at May 17, 2007 03:44 PM

Hi Len,

I am using your server code for SimpleTcpProtocolServer2 to build an application.

The application requires the server startup code to run from a different thread. I have created a callback timer to start the server. I am noticing that the server is starting and promptly destroying the thread pool with an error on the console saying:

~CThreadPool() - Shutting down worker threads

the server then terminates in that thread. All thread die.

What could cause such a behavior?

When running the server from the main thread, every thing goes OK. Any hints?

Posted by: Firas at May 21, 2007 04:02 PM

Sounds like your thread pool is being destroyed prematurely... If you need to run initialisation on a thread then you need to make sure that you either dynamically allocate all of the objects that you need to survive the end of the thread's lifetime or you keep the thread around to keep the stack based objects alive.

I could review the code for you if you want, let me know and I'll send you a quote by email.

Posted by: Len at May 22, 2007 02:30 PM

Thank you sir for your appreciated help.
I have nailed down the problem, it was an unhandled exception that forced the server to terminate. My apologies!

Posted by: Firas at May 23, 2007 12:50 PM

Hello Len,

I am developing a server application using your SocketServerProtocol2 sample application. The server writes data to connected clients. Data sent to clients is not sent on a steady manner. Data is sent as the server detects new data is there.

So, most of the time, clients will not be sent
data. Only when data comes in, it will be sent.

Is there a way to detect client abrupt disconnection using your framework?

My solution for this was to have one zero bytes read operation pending for each connection. When it completes, it is been reposted again. Is that OK or do you have any other more profound suggestions?

Thank you.

Posted by: Firas at May 30, 2007 09:44 AM

Always have a read pending for each connection. When the client disconnects you get a notification. However, remember that TCP/IP is designed to handle (and hide) service interruptions. You can only be sure that the connection is broken if you try and send and the send fails (after all of the TCP stack's retries). You can pull out cables between client and server routers and (if data isn't flowing) you'll never know (or need care)...

Posted by: Len at May 30, 2007 10:06 AM

I think, using pending reads for each connection can put out the overhead that we should maintain a socket reference if we want to keep track of idle clients (if no reads or writes are actually taking place). Right?

Posted by: Firas at May 30, 2007 02:06 PM

I found a bug in Utils.cpp. You use std::auto_ptr for arrays.

Posted by: erdbeer at February 19, 2008 06:20 PM

erdbeer,

Funny you should mention that, I found the same thing a couple of days ago when I was testing out Bounds Checker on the code. There'll be a new release of the free code eventually.

Posted by: Len at February 19, 2008 06:27 PM

Could you show us the details of the issue with using std::auto_ptr arrays?

Posted by: Den at February 20, 2008 06:51 PM

Den,

There are a few places where there's code like this:
auto_ptr<TCHAR> spBuf(new TCHAR[size]);

The issue is that auto_ptr uses delete but the new above needs delete [] to be called. This most likely results in a memory leak (only the first element is deleted). In most places the fix is probably to use std::string as a buffer, or to use the TExpandableBuffer class.

Posted by: Len at February 20, 2008 07:02 PM

std::vector is the fix!

Posted by: fixer at February 26, 2008 08:24 PM

It's a fix, yes.

Posted by: Len at February 26, 2008 09:56 PM

Hi Len,

I had been using your socket server framework for a while. Recently, I require to add an enhancement to the source code. I would like to keep the connected clients alive, until the client itself wants to disconnect or some network failure, or some time-out had occurred.

Please tell me if my approach is connected or not,

1) I can put an addref in your socket pointer and then store them per user ID after a new connection arrives. The list I maintaine will be in the socketserver base class and I will put a mutext protection everytime I use it.
Now I need to do socket->release on OnConnectionClientClose and OnConnectionReset.

Assuming no pending read and write, what is the clean way for the server to end these keep-alived connections that I held in the list? Is it psocket->shutdown good enough, or do I have to do socket->release in source code? From the shutdown function, I can't see how it go to onsocketrelease, at which I will clean my per socket data at that time.

2) I had used your CCallbacktimer in server source code, except it has one bug. The gettickcounts function does not work after 50 days. Can I just use a timestamp such as CTime and CTimeSpan to calculate the time difference and call the on-timer function when it was expired? Do you have a better solution?

Dennis

Posted by: Dennis at April 30, 2008 12:10 AM

Hi Dennis,

1) - Yes, that's correct. You need to release the socket when you're done with it and remove it from the collection. So, in OnConnectionClientClose and OnConnectionReset to deal with the connection being closed or reset by the other side and also when you want the server to end the connection. You should call Shutdown() to initiate the connection termination and then Release() because you're now done with the socket.

2) Yes, I know about the bug. See http://www.lenholgate.com/archives/000306.html for a whole series of articles about how to fix it (and how to write tests for it whilst fixing it!). The latest articles contain fixed versions of the class.

Posted by: Len at April 30, 2008 10:49 AM

hi

in CThread, I find you create thread by _beginthreadex(), but Terminate it by TerminateThread(), is there an issue? from SDK document, should Terminate by _endthreadex()

Posted by: peng at May 15, 2008 08:42 AM

_endthreadex() is a function that you call FROM the thread that you want to terminate. And if you dont call it and simply return from the thread function then it's called automatically for you (if you started the thread wih _beginthreaded().

TerminateThread() allows you to terminate ANOTHER thread; that is you call if from one thread to terminate another thread.

So, in answer to your question; No. There is no problem. There is no other way to do what we do in that code.

Posted by: Len at May 15, 2008 08:51 AM

how about your licensed version of socket server library?Such as price and...

Posted by: wintrader at June 25, 2009 05:15 PM

Hi,

I am using your server classes for one of my applications. How can I get connected client's ip-address at server in function ReadCompleted()??

Thanks and Regards,
Ajay.

Posted by: ajay at September 24, 2009 09:38 PM

Ajay,

I assume you're using the free code. If not, drop me an email for more detailed help. Anyway...

You get given the client's ip address in OnConnectionEstablished() (see the SimpleProtocolServer2 example for details of how to extract it). So, you could simply store it away at that point and use it later (I'd use 'per connection user data' to store it with the socket). Alternatively adjust the socket class to provide helper methods to getsockname and getpeername and then you can call them on the socket at any time. Note that the licensed code makes this much easier for you as the socket interface already exposes getsockname and getpeername so you wouldnt have to do anything...

Let me know if you need more help.

What are you building?

Posted by: Len at September 24, 2009 09:54 PM

Thanks for your response. I am using your free server-code in a multi-player game. I am using a buffer to record arrived-messages at 'readCompleted()'. Later, another thread processes that buffer, and may send some response back to the connected client. For this, how may I get/use 'CSocketServer::Socket' data-structure for that particular client??

Thanks and Regards,
Ajay.

Posted by: ajay at September 26, 2009 12:10 AM

Continuing my existing query, is this safe to save/use 'Socket' reference outside the 'readCompleted()'??

Posted by: ajay at September 26, 2009 04:02 AM

Ajay,

Both buffer and socket are reference counted. If you want to hold on to them or use them outside of the handlers which give them to you then you need to hold your own references. Note that you need to remember to release the references when you no longer need them. There are lots of examples of the licensed code with various servers that use reference counting to manage their sockets and/or buffers. Take a look at the documentation for the licensed code for details.

Posted by: Len at September 28, 2009 06:29 PM

Hi,

I have used your free-licensed code of the socket-server classes into a networking-software. On client-side, I am using TCP sockets to connect to the server. To avoid latency in the software, a client sends request on a unique and freshly created/connected socket each time (so, none client-side socket is reused to send another request to server). This system was running PERFECTLY on Windows Vista Ultimate up to 2 weeks before.

But, from last two weeks, same version of software gets hanged when client-side sockets started to inform WSAEACCES socket-error during Connect() call on same Vista based PC on which this software was originally built.

Later, I identified that same version of software runs PERFECTLY on freshly/newly installed Vista PC. But, same version shows WSAEADDRINUSE socket-error during Connect() call on client-side when running on freshly/newly installed Windows XP SP2.

Note that, I have test the server with 10-100 sockets in pool, 10-100 buffers in pool, and size of each buffer from 2000-25000; but response is SAME in every case.

What could be the source of problem?? What do you suggest to avoid this problem?? Thanks in advance.

Best Regards,
Ajay.

Posted by: Ajay at March 23, 2010 08:15 PM

Ajay,

That's an interesting design... I wouldn't do it that way and it's unlikely that doing it that way improves performance as each TCP connection takes time (and handshaking packets) to set up. It would be better to simply open a connection and reuse it IMHO.

Anyway... Have you installed any firewalls or virus scanning software that may have installed a badly behaved LSP?

Note that you are probably accumulating sockets in TIME_WAIT state somewhere (either on your server or on your clients) and if you create enough connections quickly enough then these will prevent you creating more - especially if you havent tuned MAX_USER_PORT.

The pooling wont be the problem. The pooling only pools the framework data structure that wraps the socket. The buffer pooling only pools the buffers that are used for I/O.

Personally I'd redesign to use persistent TCP connections... If you're really concerned about latency then creating a new connection for each request is NOT a good plan...

Posted by: Len at March 23, 2010 08:25 PM

Hi Len,

Thanks for your response.

Well, I am not sure whether my firewall or antivirus uses LSP, but our target is to support as maximum as client-systems irrespective of whether they have installed any bad-behaved firewall/antivirus.

We have changed our architecture from 'a unique socket per request' to 'a unique socket per host' as you had suggested, and the results are pretty well.

Thanks for this!!..

Best Regards,
Ajay.

Posted by: Ajay at April 4, 2010 08:24 PM

Name:


Email Address:


URL:

Remember personal info?
YesNo
Comments:


Enter this code in the box below to prove that you're not some kind of automated spam robot...