Performance Is Not Just Speed

A lot of people think performance in IT systems is all about speed. Questions like; “How fast does it go?” and “What’s the throughput?” or “How many per second?” are often thrown around. “What’s the maximum rate before it breaks?”, “How far can we push it before it crashes?” That kind of thing.

Often when people think performance, they think speed.

Performance = Speed and Speed = Performance.

And I get it; Speed is good. Speed is great.

As a performance tester, speed is one of my things.

As testement I own and commute to work daily on one of these...

It's only 800cc and not exactly MotoGP standard, but even so, I’ve been called on to answer performance-related questions by very interested Police Officers on the roadside before. Officers who mistakenly cling to a belief that distance covered and time taken are valid and the most appropriate metrics to consider.

As I’ve had to explain to them in the past; there is a lot more to consider than speed alone.

Which kind of brings me to what I wanted to talk about:

Capacity, load and fitness of purpose.

As a performance tester, capacity and load are also very much my thing.

Not only do I like things that go fast, but I also like things that carry a load.

Which is why I also own one of these things; aptly called a Suzuki Carry:

Police are nowhere as interested in having roadside chats about its performance (whether that be load-carrying ability, steadiness under cumulative acceleration, etc.) And they're still not interested even if you point out the van has a larger - 997cc - engine that generates more torque than my bike's high-revving 800cc block.

These vans are not exactly renowned for their performance capabilities, and would never appear on anyone's top-ten list of "performance vehicles".

But that's exactly my point.

My little van is extremely well suited to much higher load volumes and steady load rates than my motorbike, and because of that - in the right circumstances - it has much better performance than my bike when requirements are volume and load related, rather than pure speed.

It may not be state-of-the-art, super cool or require significant amounts of tuning by highly paid engineers, but it is cheap, economical to run, reliable and gets the job done. It's very much a work horse not a fancy high-maintenance show pony.

By focusing on one single aspect of performance - speed - people often miss the true value of performance testing as a whole.

Computer systems are not always about squeezing every drop of speed out. They should be about appropriate speed, appropriate load rates, appropriate capacity and ultimately they are about doing the right job well. Often we need computer system to be work horses not thoroughbreds.

If I pimped my van’s peak performance, installing a monster V8 it would no longer be fit for the purpose I need it to achieve. The same goes for my bike; I could add a side-car to increase its capacity and maximum load, but that would be to the detriment of everything else it does well.

It's a case of different horses suit different courses. Performance testing should test how well attuned the system is to its primary purpose, and often this has no resemblance to being as fast as possible in all circumstances. Steadiness, reliability, robustness, recoverability all come into play.

To me, any computer system’s primary purpose is to be used by - and to be useful to - humans. They should display Human Quality. To give this its rightful performance slant, I believe being useful to humans means systems must behave reliably, be robust and must operate as smoothly and quickly as we need them to.

Only some of this relates to speed.

And that’s why I love my little van (almost) as much as I love my bike. It's my high performance vehicle of choice.