The importance of defining performance goals

This post was originally created in the Productive C# blog in 2016.

Let’s start from some questions:

• Is you application performance unsatisfactory?
• Are your customers complaining about your application performance?
• Is your business demanding more performance to increase sales?

If the?answer is yes, it’s likely that your manager asked your team to spend some time to improve the performance of your application.

Most of the time, the goal is to generally improve the performance or an ambiguous sentence?like?“We need to be able to reliably support at least 100 users”.

As engineers, we are often excited?about this kind of opportunities as making a system more efficient?is usually fun.

Some of you may?already have an idea of an area of the code that likely is the cause?of the problems and they can start immediately tweaking and optimizing the code.

Others knows that measuring first is important. They are a bit more cautions and they run a profiler, measure the CPU usage or memory and use the tool to identify a bottleneck before trying to change the code.

Others challenge the assumptions of their colleagues and demand to run the tests in a more production-like environment. So they focus first in building an?infrastructure before even starting to run some tools and measure performance.

Are you familiar with it??Does this sounds reasonable?

At the end,?improving the performance of a software application can be hard. It’s kind of expected that there?will be a discovery process, some research and trying different things hoping to identify and fix the?major limiting factors.

You know that?improving performance is all about measuring! Measure measure measure is key. We measure, we try to make some changes and hoping to see some benefits. Sometimes however we feel we are not making any progress or the changes we are making are not having a big impact.

What is going on?

The answer is simple…

You are wasting your time!

There is no point in measuring performance or using profiling tools if you don’t know what you want to measure and what performance goals you want to achieve.

Of course, the performance goals depends by your application and domain.

You need to spend time with you team and your customers in thinking?why you want to improve performance?and?what are the business goals you want to achieve.?This is not an optional step.

Setting general and ambiguous goals is meaningless.

A performance goal is expressed in terms of a quantifiable performance metrics that can be measured by some means of performance testing.?Classic performance metrics are?CPU time, Wall-Clock Time, Memory, Latency, Throughput,?Disk I/O but you can define your own metrics that are specific and only relevant to your project.

It is also extremely important to?define the environment that?will be used to test performance.

Finally,?every single word used in a performance goal statement must be carefully defined.

What a good performance goal should contain?

A?good performance goal should contain:

• The?benefits for the customers and the business
• The key quantifiable metrics that need to be measured and their expected values
• A well-defined environment
• A clear explanation of the meaning of every word used

This is an example of a bad and unclear performance goal:

The server CPU utilization should not exceed 75%

This is the same example expressed in better?terms:

The CPU utilization of the server?over a period of 1 day should not exceed 75% when we monitor 50 machines?each running 2 SQL Serve 2014 instances with 10 databases…..

Often a sentence is not enough and a table can be a better way to express the goal. It’s up to you to decide how to express the goal.?What it matters is that all the elements of a good performance goal are present and that the goal is crystal?clear to every project stakeholders.

With a good performance goal the engineers know exactly what to optimize for, what sort of environment and resources?are needed to reproduce the problem and can set up a reliable system to measure the effects of their efforts (possibly automatically).

This is the call to action:

Stop doing random?optimizations and focus on defining?clear performance goals first.

The following list of questions can help to do so:

• What specific?performance problems are?customers reporting?
• Why do we want to improve performance?
• What exactly do we want to improve?
• What are the key performance metrics to measure?
• Are our goals and metrics quantifiable?
• Do we have a clear definition of the terms we used?
• Have you defined a representative environment?
• Do you have the resources to create such environment?
• What kind of workload should we use in?our performance test?
• Can you show how close you are to met the?performance goal?
• Can you automate the performance test?
• What is the optimal theoretical performance you can achieve?

Resources:

• Writing High-Performance .NET Code
• Pro .NET Performance: Optimize Your C# Applications (Expert’s Voice in .NET)
• Pluralsight: Measuring .NET Performance