Socket Server code updates

| 102 Comments

The latest release of the free version of my asynchronous, windows, IOCP based, socket server framework can now be obtained from here at ServerFramework.com.

For some time I've been promising to update the socket server articles to use the latest version of my code. Today I finally updated the code for the first article. I'm going to update the article itself soon, but in the meantime I'm posting the new code here.

[NOTE: This code leaks when adjusted to build with Visual Studio 2005. This seems to be due to a bug in VS2005's STL. See here for a workaround. I expect to post a fix for this once I've found time to convert the code to build with the new compiler and then tracked down the leak. If anyone has already done this then a comment telling me what's wrong would be appreciated.]

What's new?

1) You can now compile the code with VC6 without a Platform SDK installed. To do this, edit NoPlatformSDK.h and uncomment the #define.

2) The sockets now share critical sections. The previous code suggested either allocating a critical section per socket, or sharing one section between all sockets, at the time I mentioned that this was less than ideal and that you'd probably want to do it differently in a production system. Now I'm including a shared critical section system that uses a hash to spread the contention across a pool of critical sections.

3) Servers can now share the IO thread pool. You can now create multiple servers and have them share a single pool of IO threads. This makes creating servers that listen on multiple ports much less resource intensive.

4) The server can now initiate outgoing connections as well as accept incomming connections.

5) All examples can now build with VS.Net 2003 as well as VS.Net 2002 and VC 6. All examples come with project files for all three build systems.

6) You can now have multiple user data "slots" per socket. This makes it easier to layer services on top of the server and still allow the client to add their own user data to each socket if they want.

7) Each example ships with a test harness and config file to test the server as part of the build process, if required. Notice the difference in performance between the debug and release versions...

Code is here.

102 Comments

Thanks a lot.
Your code is very help to me.

Thanx for your code, very nice work. I have tested it for quite a long time, some problems unsolved that may be a configure inaccuracy or some performance bottleneck somewhere, the issue is: I always confront intensively large ratio of denied connection from client side, say, a test of 500 threads, 40% ~ 90% of connections will be refuesed, although I may trust your architecture as optimized for large-scale network server, but such high ratio of failed connection may suggest its inability to handle high-load connections. I have tried to change socks in pool and way of threadpool, but no noticable improvement. I hope you could possibly reply me on this problem, and if convinient, give suggestions on high-load server. Thanks!

Have you tried running the test from more than one client machine? It may be a problem with your client side code, or it may be a problem with the client side using up all available ports if it's opening and closing connections quickly (leaving ports in TIME_WAIT state, etc). What does netstat show on the client and/or server? Are you running the release version of the code? The debug version is considerably slower due to the locking done around the debug output (this makes a huge difference to the performance of the app).

Thanks for reply, yes I have a carefully arranged test so all possible situations, include local test and between workstations, and the client tools are vary also. The result is similar, though we might go around with less chan se of error by turning down the numbers in:

300
10
1000

but anyway the disability to connect to remote socket did happen in most tests, combined by an other error that indicate a failure to transport any message. I havent found a solution to it yet.

An other problem is the memory issue, it seems quite interesting, the echo server and packet server are using extremly little memory that is comparable to virus, among 200k-400k, when under large connection pressure, and it decrease memory usage as time goes by, very strange. On the other hand, the large packet server with thread pool eats more than 300M memory when it starts to take incoming connections, abnormal, might be leaks somewhere.

sorry, 300,100,1000 are actually settings, xml tag are filted after post. I forgot to mention the block on CSocketServer::OnConnectionReset during one test method in which I send request to 1000 local http proxy servers to connect to the test server running jetbyte socksvr, repeatly, to simulate real-world server task, and I found, very soon after several sessions, it will hang up in onConnectionReset, the version I used is the updated echo server. Hope u could give useful advise, greets!

What does your client machine's netstat command say at the point where you fail to connect? I expect you're running out of ports and ending up with lots of old connections in time_wait. This is normal - you dont expect to run 10000s of short lived connections on the client side. As I suggested, run with multiple clients using fewer connections per client and see if you can go above the limit for a single client. If so then I'd suggest the problem is on the client end and not the server end.

The memory issue is as expected. Take a look at the code... The echo server uses 1024 byte buffers and writes the response back to the client using the same buffer that it uses to do the read (as far as I can remember). Memory will decrease as we only pool a certain number of buffers, so when the connections die the buffers are released. So it will only use 1024 x number of active connections for buffers. The large packet echo server uses 1024 byte read buffers and 65536 byte message buffers (between socket server and business logic thread pool), it also echoes the messages back in small chunks (666 bytes) so will naturally create many more buffers than the other servers. This is by design, read the associated articles for the reasoning.

If you need masses of incomming client connections then it's probably worth specifying the 'use zero byte reads' option as this will reduce the amount of 'non paged pool' that is used by the server. Again, read the articles and comments for details.

thanx for your answer.

Thanx for your greate code, it was very useful to me about basic of IOCP architecture.
but, i have some request..if you will.
I wondering belongs -the Test.dsw code include whole project..

-SocketServerTest.exe
-SimpleProtocolServerTest.dll
-SocketServerTestInterfaces.dll

because, As i made custom server use your library,I don't have especially testing Tool.
so, Could you show me some test library code or let me know how do you made the TestingApp.
For example,
Testapp's architecutre?
The sequence?
the way..

got it? My english is very poor. sorry... :)
thankx.

Thanx for your greate code, it was very useful to me about basic of IOCP architecture.
but, i have some request..if you will.
I wondering belongs -the Test.dsw code include whole project..

-SocketServerTest.exe
-SimpleProtocolServerTest.dll
-SocketServerTestInterfaces.dll

because, As i made custom server use your library,I don't have especially testing Tool.
so, Could you show me some test library code or let me know how do you made the TestingApp.
For example,
Testapp's architecutre?
The sequence?
the way..

got it? My english is very poor. sorry... :)
thanx.

The test harness is written in C#, it's documented in this article, source is supplied: http://www.jetbyte.com/portfolio-showarticle.asp?articleId=46&catId=4&subcatId=11

Your english is far better than my Japanese (guessing from your .jp email address) :)

^^
thanx, very quick..!!! :)
and also, you should erase i wrote comment in duplicate.

ps.your home page is very smart~!
i like your style. ^^

i have question one more time..

in XML
i can't type ^,^ so, replaced "[","]" ok?

[Conversations]10[/Conversations]

why do you have added the parameter, if i had set up

[Multiple]>true[/Multiple]

then, do i need set up

[Conversations]10[/Conversations]

how about it, the code have modified to

[Conversations]1[/Conversations]

and, in plug-in dll , added more message
alike

messages = new IMessageExchange[100];

messages[0] = new MessageExchange("USER test\r\n");
messages[1] = new MessageExchange("PASS test\r\n");
messages[2] = new MessageExchange("STAT\r\n");
messages[3] = new MessageExchange("RETR 1\r\n");
messages[4] = new MessageExchange("RETR 2\r\n");
messages[5] = new MessageExchange("RETR 3\r\n");
messages[6] = new MessageExchange("QUIT\r\n");
.
.
.
.
.
messages[99] = new MessageExchange("QUIT\r\n");

it was almost nothing to do with server testingApp.
but, i just wonder, is it right? or not.

The number of conversations is the number of times the entire test is run on the connection. So the test harness will connect to the server, it will then loop for the number of conversations and each time through the loop it will send all the messages that the MessageExchange object contains.

The multiple message flag is used to decide if the server will send multiple messages in one send(). This forces your server to deal correctly with multiple messages arriving in a single read call... The fragments flag determines if the test will send bits of messages and force your server to deal with a single read not returning a whole message.

oh!!!!!
you mean, you wanna test just connection per thread... after all, one thread is almost same that seperated computer or client.
is correct? i understood,your testapp was just Send/Reply Testing.. i miss it.. right?
is Conversation parameter for just connection testing?

What & how should the server do, while the client pc is shutdown or turned off.

The server may or may not realise that the connection to the client has been terminated; read up on how TCP/IP works. If the client or server closes the connection cleanly then the server will know straight away. If there's some sort of problem (network cable pulled out, router failure, client pc crashes) then the server will only discover it when it next tries to send a packet to the client. If you want to know as soon as possible then you need to either use the TCP/IP keep alive stuff or implement your own application level keep alive protocol.

Re the test harness:

I was wrong with my description the last time around.

The test runs with multiple threads. Each thread runs a test. Each test simulates multiple connect/disconnects each of these is a 'conversation'. Within a conversation the messages in a message exchange are sent/recieved.

I wanna ask question related to following constructor function...

CSocketServer server(
lockFactory,
pool,
"+OK POP3 server ready\r\n",
INADDR_ANY, // address to listen on
5001, // port to listen on
10, // max number of sockets to keep in the pool
10, // max number of buffers to keep in the pool
MAX_RECV_BUFF); // buffer size


what's the limit?

socket number, buffers, buffer size?

I knew, it's depend on myServer resource.
but, i wanna know what things have something do to with the parameters.

Could you explain ... concern with client(user),
for example,
i waana service for 1000 user
max packet size is 1024*10 byte = 10Kbyte
then how many buffers i need? set the case, hardward(server) resource support enough.

The numbers for the socket and buffer pools just relate to how many socket and buffer structures are kept in memory ready for use. If you need 1000 buffers then 1000 will be allocated, if you set the buffer pool size to 100 then when all 1000 buffers are no longer required you will still have 100 in the pool ready for immediate use. It's a performance optimisation...

i having implement own chatserver,
i try modify your SimpleProtocolServer to simplechatserver.
but, On broadcast active user,
Can i use
SocketList m_activeList;
?
but, this is protected member,
then Do i have to create some method for get m_activeList.
if have idea, please explain..

ps.
hey len, didn't you sleep?
what time are there?

Sorry to disturb you again but i like to clear one problem in my try...
i found your document in codeproject.com
there are some hint about hadling connections.

//////////////////////////////////////////////
Maintaining per-connection state
The final thing that our server may need to do is associate some internal server state with a particular socket connection, the Socket class makes this particularly easy as it provides the following member functions:

void *GetUserPtr() const;

void SetUserPtr(void *pData);

unsigned long GetUserData() const;

void SetUserData(unsigned long data);

/////////////////////////////////////////////

so, to make chatserver-broadcast function- should i use above function? if so, Do i need other criticalsection for connection manage?
plz give me some advise.

thanks.

after all,
in a word, where do i get connected user list?
thanks. again. :)

sincerly yours.

Yes, use the Get and Set UserData functions. You need to make sure you tell the server how much space you need, you pass a user data slots parameter to the server constructor. You probably dont need to lock around access to the user data slots as normally only one thread will be doing this at a time...

Re the user list. I'd personally implement my own application level user list. I wouldnt try and use the active socket list that the server maintains.

--) Re the user list. I'd personally implement my own application level user list. I wouldnt try and use the active socket list that the server maintains.

the meanning.. is it my own job?
to get the userlist, Do I have to own application level user list for broadcasting message? right?

sync.(criticalsection for userlist) as well?

my english is so poor, so it's difficult to know what you waana explain..

sorry..

Yes. You need to write the user list and manage it. You will need to use a critical section to sync it.

Your english is still better than my Japanese, don't worry about it :)

Hello Len

Great code.

Have you compiled and posted the COM component yet? Also how do you code a client to talk to the server.


Regards

JB!

Joseph

I haven't updated the COM component; truth is, we don't use it for much so it doesnt get much attention. It's on the list though.

Quite often we use the server framework for clients as well - hence the outbound connection support. It depends on what the client is for and which platforms it supports. Other times we use something similar to the MFC client that we put together for the SSL article.

hi.there.
it's me agian. didn't you miss me? ^^

I modified your code for chatting server.

in Sample simpleprotocolserver,
i have chaged the code,

void CSocketServer::ProcessCommand(
Socket *pSocket,
const CIOBuffer *pBuffer) const
{
.....
....
...

if (ok)
{
// We understand, but we aren't really a POP3 server...

std::string response("-ERR sorry, we understand \"" + echoCommand + "\", but, we're just a fake POP3 server...\r\n");

{

JetByteTools::Win32::CCriticalSection::Owner lock(m_listManipulationSection);

Socket *pSocket = m_activeList.Head();

while (pSocket)
{
Socket *pNext = SocketList::Next(pSocket);

pSocket->Write(response.c_str(), response.length());

//pSocket->AbortiveClose();

pSocket = pNext;
}
}
...
...
...

}

i also have chaged privte member value
"m_listManipulationSection"
"m_activeList"
to public member

any problem?
I didnt follow your suggestion. but, it was not problem, yet.

Can i use this way, for chatting server?

thanks.always.

> any problem?

You're coupling your code to the framework in a way that wasn't intended. It'll probably work. I'd probably do it differently.

hi,there.
i have made newproject using Devstudio appwizard.
project-win32application (vc6.0)
and, your simpleprotocol souce file move to DSW.
and having compile.
but, there are many many linking error occur.
- LIBCD.lib crashed...
so, i ignore LIBCD.lib in my project option, but,
another error occur.
-LNK2001: unresolved external symbol _errno

so how can i resolve the problem?
and there are some lint option, but, i don't konw what it does.
thanks.

Sounds like your compiler options in the new project dont match the library options. You need to make sure that the new project uses the multi-threaded static library build, not the dll or single threaded build.

great!!
thanks. anyway, it's too difficult proejct setting like this..
I often encounted such problem..when i linked C library and anything else.

thanks.

I never seem to create new projects these days, I copy a dsp file, open it in notepad, do a search and replace for the 'blah' of 'blah.dsp' bits and then open it in VC and adjust the files. This way all the settings stay the same ...

one more...
you seem to be forget my another question.
about "lint option"..
could you explain for me?

The lint options file and lint comments are because I use Gimple Lint (http://www.gimpel.com/) which is like running your compiler with the warning level set to 11.

I don't know if I like the fact that I contaminate the source with warts that suppress warning messages for things that I want to be warned about but sometimes like to turn off because I know that the warning is being too cautious... It works, but I think I'll end up moving away from the //lint comments etc and use external suppression files.

Hi Len,

Thanks so much for sharing your code. Out of all the free packages I have seen, yours is by far the most well designed. Unfortunately, when just testing the code on my laptop I can not ge the echo server to respond reliably. I am assuming that this is something about my particular setup, but I just wanted to let you know.

I am running an IBM Thinkpad P3 500 with Win2k Pro, MSVC++ .NET (2000) and the loopback adapter. I was just running the compiled version and connecting with telnet.

Thanks again for sharing your hard work with all of us.

Cheers,
Adeh

Is this the C++ echo server you're talking about or the COM one? Try changing the ports? I think 5000 is used for something on some boxes. Try doing the complete build and letting the test harness run? If it's the packet echo server remember that you have to send the packet length first (see article).

I know it's not in any way helpful to you but 'it works for me and the test harness thrashes it pretty hard', I'm surprised you're having problems with it...

your code is very good

i want setup business server support more than 10000 client from diffent workstation on internet.
your code is usefull to me.

Aku - Thanks. Do you mean 10000 simultaneous client connections? If so you probably want to set the 'post zero byte reads' flag when constructing your server. This reduces the amount of "non-paged pool" that the server uses when it's running.

thanks.

i setup a project use MFC APPWizard. than add your
Win32Tools to the project, but report errors

Linking...
nafxcw.lib(afxmem.obj) : error LNK2005: "void * __cdecl operator new(unsigned int)" (??2@YAPAXI@Z) already defined in LIBCMT.lib(new.obj)
nafxcw.lib(afxmem.obj) : error LNK2005: "void __cdecl operator delete(void *)" (??3@YAXPAX@Z) already defined in libcpmt.lib(delop.obj)
../../Bin/OutlookBar.exe : fatal error LNK1169: one or more multiply defined symbols found
Error executing link.exe.

how to resolve the problem?

my project Use MFC in a static library.

Looks like you're not linking with the multi-threaded libraries. The socket server stuff links with the multi-threaded versions of the runtime libs so your main project also needs to link with these.

The socket server stuff links with the multi-threaded versions of the runtime libs so your main project also needs to link with these.

Yes, main project link with the multi-threaded versions of the runtime libs , still report that.

Hi Len!
I'm sorry for this real quick q. I already sent you an E-Mail about it. So. I based my project on the article "Handling multiple pending socket read and write operations" and the code there "..server6". Now I see you have an update. but all the projects in SocketsServer.zip look a lot different from what is in the article. For example the thread pool instead of like 10 params has just one :D. On the other hand I see bits of the code in socketserver.cpp that handle the sequence. So the question is which project in the socketservers.zip is the one that takes care of the "Handling multiple pending socket read and write operations"? Can you just quickly give me an idea where at least to read about this package?

hi len, how are u?

i resovled problem above, your simple protocol is too simple, i must write a protocol ,
^_^, that is very simple to me. thanks you very much.

happy new year, have a good time.

Max, the code has been updated, the articles haven't. Therefore you should look at the code that comes with the articles if you want to understand the code and then compare to the latest code if you want to understand the changes and fixes. I haven't had a chance to update the articles yet.

hi, len.
when did you update your lib?
you mean, since I ask some problem above, you did?

The code hasnt been updated since I originally posted this entry.

Leave a comment

About this Entry

Developer buy in was the previous entry in this blog.

Slacking is the next entry in this blog.

I usually write about C++ development on Windows platforms, but I often ramble on about other less technical stuff...

Find recent content on the main index or look in the archives to find all content.

I have other blogs...

Subscribe to feed The Server Framework - high performance server development
Subscribe to feed Lock Explorer - deadlock detection and multi-threaded performance tools
Subscribe to feed l'Hexapod - embedded electronics and robotics
Subscribe to feed MegèveSki - skiing