Can performance tests treat the object under test as a black box?


Barry suggests that to do meaningful performance tests you need to know a bit out the way the thing that's under test operates.

I guess he has a point given his reason for performance testing was to compare a new version of the thing under test with an older version of the same thing...

Personally, I'd be tempted to leave the poorly performing tests in the test harness. Then add some comments about why these particular situations are unlikely to show up as real usage patterns and why the object under test performs poorly in these particular situations. Think of them as documentation that compiles... ;) If usage changes, or is, in fact, different, then you have some tests that will help as you refactor your way to improved performance.


It's an interesting case with the tests of the performance rather than the code itself that I can't really write the code to be self documenting, i.e. PreCachingRequest(); followed by DoMultipleRequests(); because if the behaviour of the server changes the test code doesn't make any sense (a bit like #define RED 0x00ff00 // green!!). The tests all time and report back their results ;) - in fact my precaching test actually shows that my code does that part in around 10 seconds, versus 100+ seconds for the old C-server.

In plain language the precaching bit could be part of the server's start up routine, but since it takes so long in the current C version, the caching is done on a per request basis, whereas mine could do the caching as part of the start up....

Currently I just have trusted std::cout telling me which bit is doing what and why....

Dear Len,

I like to ask you something, please tell me if you have some sort of info about the topic.

I've worked a lot on Winsock. But this work is limited only to transfer text. I like to work on applications relating to transfering voice, calling from PC to phone e.t.c

But the problem is that from where to start. I am in search of some "good book" that details about voice packets, formats, how to transfer voice e.t.c

Can you refer me some good book. Yes there are tons of books on VOIP topic available from Amazon. But i only require a good, single elementary book. It is difficult to judge which book is good one from Amazon, people have different opinions about a single book.

Can you tell me from where to start, or some god book on related topic or whatever advice you like to give.



I've no idea, sorry. I've not done any VOIP work. I was browsing some of the VOIP books in Borders yesterday though and most of them seemed aimed at a 'managerial' level and appeared to be trying to sell a particular companies hardware, etc. I didn't see anything that seemed especially technical.

Leave a comment