Can performance tests treat the object under test as a black box?

Barry suggests that to do meaningful performance tests you need to know a bit out the way the thing that’s under test operates.

I guess he has a point given his reason for performance testing was to compare a new version of the thing under test with an older version of the same thing…

Personally, I’d be tempted to leave the poorly performing tests in the test harness. Then add some comments about why these particular situations are unlikely to show up as real usage patterns and why the object under test performs poorly in these particular situations. Think of them as documentation that compiles… ;) If usage changes, or is, in fact, different, then you have some tests that will help as you refactor your way to improved performance.