The IT Galactic Series - Journey to Effective Performance Engineering

In the IT Quality Assurance (QA) world, some of the key criteria that are often mentioned include Availability and Resilience of Systems. Despite the importance of these criteria, performance testing is a technical testing discipline which is traditionally often overlooked and not budgeted adequately in many organizations. With the evolution of digital transformation and DevSecOps, performance testing has gained more relevance and is nowadays being driven under a wider scope of Performance Engineering. New initiatives such as Chaos Testing are becoming more and more popular as well due to accelerated delivery cycles. This articles aims to debunk some confusion on performance related terms and share some best practices that can assist organizations in driving performance engineering effectively.

Here's my favorite summary of the evolution to Effective Performance Engineering written by Todd DeCapua and Shane Evans:

"While Performance Engineering is often defined narrowly as ensuring that nonfunctional requirements are met (such as response times, resource utilization, and throughput), the trend has moved toward a much broader application of the term.

“Performance Engineering” doesn’t refer only to a specific role. More generally, it refers to the set of skills and practices that are gradually being understood and adopted across organizations that focus on achieving higher levels of performance in technology, in the business, and for end users.

Performance Engineering embraces practices and capabilities that build in quality and performance throughout an organization, including functional requirements, security, usability, technology platform management, devices, third-party services, the cloud, and more.

Stakeholders of Performance Engineering run the gamut, including Business, Operations, Development, Testing/Quality Assurance, and End Users."

What is the difference between Performance Testing and Performance Engineering?

Since the term performance engineering is often misconstrued with performance testing and performance monitoring as well, this is how I clarify the confusion with clients in a simplified way:

Key Practical Steps to engaging in Performance Engineering:

1. Collaborate. QA can play the role of champions of Performance Engineering in the Modern IT world but, it is key to understand that many players have to be involved in this process and that it is a collaborative initiative. One can then successfully ensure that the right performance engineering requirements are documented from the initiation of any projects or releases.
2. Shift Left. We must drive the shift left approach of performance testing validation and verification activities by requesting the right level of collaboration with developers, architects and operations team. One should plan for performance testing from a component level, API level, system level, system integration level and business process level .
3. Maintain easy to use test packs. Don't lose sight of the importance of performance regression and sanity packs. They assist in ensuring that a new baseline system is as effective as the previous baseline system. With sanity packs, one can leverage them to do daily automated health checks in different environments.
4. Monitor your Metrics. Review your key metrics that you want to drive for effective reporting and decision-making. It can include throughput, response times, concurrency, errors per sec, hits per sec, etc. However, any good performance test should include additional monitoring metrics to correlate with the tests that you execute.
5. Shift-Right. The Shift Right movement aims to have a more interactive bridge with real user and infrastructure statistics from Production into your performance strategy. It is about leveraging production monitoring back into your SDLC cycles and also driving some level of performance testing (ex: sanity tests) in Production through controlled mechanisms.
6. Technology Inventory. Understand your toolset coverage. Since Performance Engineering requires a lot of automated testing, it is important to understand the difference and value of each performance related tool from performance testing to performance monitoring. You can also leverage automation testing tools, integration testing tools amongst others.
7. SDLC in context. Adapt your processes onto the SDLC model in use in your organization. Business, IT and organizational context are very critical to the success and sustainability of your approach and strategy.

Lastly, it is key to understand the maturity model for your performance engineering journey. It empowers you to be able to manage your customer expectations more efficiently and be more realistic in your goals.

I hope you enjoyed this little snippet about performance engineering. Thanks for your time!