Quality Engineering for CEOs: How Much Should You Spend? (Post 2)

TL/DR

• Software engineering personnel spend should generally drive quality resources spend
• Companies can target 10%-20% of engineering personnel costs for quality resources
• Six proxies are helpful: warranties, insurance, wellness, maintenance, tips, and bonuses
• Factors such as age, tolerance, coverage, and expectations often influence spend?

Introduction

In my first article, I introduced my background (30 years in software). I provided insights into my previous disappointments with quality assurance. I offered how things have changed during my journey with Testlio. And I advocated for a strategic approach to quality engineering for software-centric businesses that warrants CEO-level consideration.?

Today I tackle the first QE-related question that many CEOS, like me, often ask: how much should we spend on software quality resources??

For a shortcut, my initial answer—with no additional context—would be: “We should spend 10%-20% of our software engineering personnel costs on focused quality resources.”

While some experts argue that quality spend should be relative to company revenue, I disagree. I hold that the better corollary is software development personnel costs. Before we unpack why this is a reasonable driver, and how we might think about it, let’s explore what it means.?

Understanding Quality Spend as a Percentage of Engineering Costs

For many companies, software development costs are 80%-90% people-based. Yes, hardware, tools, environments, access, and other expenses come into play. But for most organizations, people are the largest cost-driver of the engineering function.?

This article will focus on quality resource (people) costs as the primary driver of overall quality spend. How then might we estimate a fully-burdened cost per engineering head (FTE)? Today, the cost of a software developer varies greatly. Location, experience, technologies, and other factors come into play.?

For simplicity purposes, we will use USD $10K/month as an estimate of the blended, fully-burdened cost of a full-stack software engineer (noting that costs can be higher, especially in high cost of living areas, but are lower in many parts of the world).

So, for a team of 20 developers, a company may spend $200K/month, or $2.4M/year on core software engineering people-related costs. Backing our way into 10%-20% on quality resources, we land at $20K to $40K/month on quality spend. With $30K (15%) as a midpoint, the company would target $360K/year spend on dedicated quality resources.

You’re probably wondering: does 15% actually make sense? What’s it based on? And what factors might drive it up or down? We’ll get there. But before we do, I will offer six proxies across three categories for comparison purposes.

Real World Proxies for Thinking About Spending on Quality

Stepping back, let’s give ourselves permission to play with six thought experiments. Some of these are more of a stretch than others. But stay with me as there appears to be a pattern.?

Quality Engineering as Risk Mitigation

I sometimes conceptualize quality as a means to minimize business financial risk due to production bugs. In this case, let’s explore how the quality function acts like an extended warranty or insurance policy.?

Neither are exact corollaries, as both are arguably about mitigating financial outflows after something has occurred. Quality engineering is (mostly) about minimizing the potential for issues occurring in the first place. Yet these concepts do provide guidance: a rule of thumb on extended warranties is that they often cost 10%-20% of a product.????????

Insurance policies are another reasonable point of comparison. As of 2024, a new car in the US costs around $50K. Insurance is approximately $1.5K/year. The average American owns a car for 8 years, making insurance ~$12K—or 24% of the car's initial value.

It’s not by accident that the market prices warranties and insurance at these levels. A 10%-20%+ price range has proven to be a viable risk/reward trade-off for suppliers and purchasers over extended periods of time.

Quality Engineering as Issue Prevention

Per above, the point of quality efforts in software development is generally to prevent bugs. Two related avoidance concepts are personal wellness programs and home maintenance services.?

The US wellness industry is worth approximately $1.8 trillion per year. For a population of around 330 million, this amounts to just over $5,000 per person each year. It’s tricky to determine the personal value of this spend. For many of us it’s about quality of life, longevity, and/or disease avoidance. At least for some of us, spending $5,000 to avoid $25K to $50K (10%-20%) of downstream medical costs, and/or lost personal experiences, would be a worthy tradeoff.??

Home maintenance is also a potential parallel. Many advocate budgeting 1% of the value of a home each year for maintenance. Over 15 years, this would be 15% of the home cost. Another approach is to take 10% of the monthly mortgage, taxes, and insurance costs. There's a bit more math here, but the formula leads you to a similar place over the lifespan of the home (e.g. 15% if you stay in the home for 15 years).

Quality Engineering as Experience Excellence?

Some software quality initiatives, especially in the realm of usability, exploratory, accessibility, and performance testing, can be thought of as going beyond issue avoidance (catching bugs) to ensuring end-user joy, inclusion, success, and promotion.?

Let’s explore tipping and bonuses as comparative areas. Whatever your feelings are on the growth of tipping in places like the US, the multi-decade standard for tips is 20%. I was born in 1970 and originally trained by a father who insisted that the tip was there to help ensure that a person provided exceptional service. Might we think of a portion of a 10%-20% spend on quality in a similar way—that it helps drive outstanding user experiences by motivating quality experts to push for excellence??

Bonuses and other forms of variable compensation are also used to help drive results. In the US, the average worker bonus is ~10%. Variable compensation can be higher depending on role (e.g. sales), industry, level, and other factors. While intrinsic motivation is powerful, many companies use extrinsic rewards like bonuses.

The 10%-20% Pattern

The numbers above are based on a set of simple Google Searches that surfaced multiple indicative, clustered results. My attempt here was not to be exact. Yet I do offer that across six proxies (warranties, insurance, wellness, maintenance, tips, and bonuses), a pattern of 10% to 20% emerges.?

Whether or not you agree with the concepts or math above, I’m curious about what you spend at your company today on quality resources? Might you find that you’re in the 10%-20% zone??

I talk to engineering leaders, product leaders, CEOs, and CFOs regularly. I find that this 10%-20% range is prevalent.?

That said, if you’re significantly above or below this range, should you be concerned? And if you’re uncertain as to how to think about your targeted spend level, what other factors might you consider???

Above I mentioned that my guidance on quality resource spend is 15%—with no context. As you’d imagine the real answer is more interesting. Let’s end today’s article with an exploration of four factors that can influence your quality personnel economics.

?Quality Resource Spending Factors to Consider

Software Technology Age

In general, the age of a software product correlates with complexity, breadth, and tech debt. With age, software products can become harder to assure. My personal recommendation would be to consider the following:

Issue Risk Tolerance

Regulations, competitive environments, company cultures, and other factors can drive the level of risk tolerance for production issues that an engineering organization holds (and yes, risk tolerance can decrease with age too, per above). A simple heuristic for risk vs spend would be:

Testing Coverage Requirements

Some software products must work across dozens of countries (locations), languages (translations), devices (hardware), and payments (methods). As you can imagine, the more versions of your software, the more you should generally spend on quality. Here’s a rough guide:

Engineering Quality Expectations

CTOs and VPs of Engineering have multiple options when it comes to quality. Some of my favorite engineering leaders have no dedicated quality resources. Based on their philosophy, culture, systems, economics, and technologies, their software developers own quality holistically.?

This approach generally isn’t for the faint of heart (especially with age, coverage, and risk considerations outlined above). It can impact engineering velocity, innovation, and economics. But it is a viable strategy, and in the right conditions it can be powerful.???

Combining Quality Spend Factors

You may be wondering: so, could I calculate a weighted average across the four factors above? My personal sense is: absolutely you can!?

For example, if we take a simple equal weighting (25% per factor), a 3-year-old software product (10%) with medium risk tolerance (15%), broad coverage requirements (20%), and significant engineering quality expectations (10%), would conceivably drive a quality resource budget of 13.75% of software development personnel costs.?

You can try the above with the specific characteristics of your company, team, and product to see what the formula offers as guidance (noting that this is meant to be a rough proxy, not an exact science).

In Summary and What’s Next

Today’s article sought to build on the first article’s key point—quality engineering should have CEO-level attention in software-centric companies. Exploring six proxies, it proposed that companies should generally spend 10%-20% of software engineering personnel costs on dedicated quality resources (people). It then examined four factors that can influence spend.

So, where do we go from here? In my introductory article, I laid out 12 question clusters that I would attempt to answer. My instinct is to next go to the last question on that list:?

What does a great quality strategy entail??

With this, I’ll use a definition of strategy that is derived from Michael Porter’s work in the 1990s. “A strategy is a set of activities that an organization undertakes to gain a competitive advantage. Strategy should dictate fundamentally how we drive our investments and resources. With a sound budget and a smart strategy, we can drive the success of our companies.”

Intrigued by how strategy might apply to quality engineering? Please come back next time. And let me know what you think about today’s article via the comments section here on LinkedIn. I took a few logical and mathematical leaps and would appreciate your feedback.