Defining the KPI for Quality Analysis

Introduction

• Recently I have attended the session given by S Reine De Reanzi IIMB, PhD on “Community Discussion on Defining KPI for Support Team” which focuses on 2 primary questions.

1. Is QA responsible for all the misses?
2. How do you find the health of the application?

• Both the questions are very important for self-evaluation for all of us which leads to think how we can prepare our defence as a QA based on above questions.
• Gathering and formatting the data points and presenting them in a structured format is a way and here comes the need of KPI (Key performance indicator) or Metrics for quality measurement.
• KPI can be for all stages of the development cycle, but we would be discussing two main areas:

1. Delivery
2. Post Delivery, Released Functionality

Try to find out Solution for above questions.

Is QA responsible for all the misses?

• Before answering this question, we may need to answer does QA involved in all the stages of the software development?
• If No, how does QA is responsible since Quality in terms of defect is Developers responsibility, Quality in terms of feature is Product responsibility, Quality in terms of uptime is Infra guys or devops responsibility, where does QA owns quality?
• People often come back saying QA owns overall quality but if this is true, QA should be involved in all the stages with all the information shared with them.
• Lets put up the process and KPIs to see if the QA is only person who is responsible for this or as a team everyone plays role

How do you find the health of the application?

• Health of the application can be divided into two parts.

1. Infra health (Uptime)
2. Functional Health (Error free to use)

• Let’s focus on 2nd point and see if we have anything to measure on the same.
• There are lot of things we can gather and present in order to see if the 2nd point is going well.
• Please check the below mind map which shows the KPIs to gather overall but lets more focus on defect leakages.
• If the defect leakages are about 13 to 15 %, it is as per the industry standards but there is a scope of improvement, but then other items should be looked at if we are not achieving the quality. (Other things are mentioned in the mind map below)

Controlled Environment KPI

• There are 2 very important KPI for measuring the quality of the software in the controlled environment.
• Controlled environment means when we have access to code and make changes to find out if our test suites have capabilities to find the bugs.
• Mutation testing plays an important role while calculating the same since we need some numbers on the basis of which the calculation could happen.
• Mutation score and APFD are very important and will measure the effectiveness of the test suites and how many defects/mutants it kills.