Productivity, Test Effectiveness and Test Efficiency Metrics and KPIs for QA Professionals

I see multiple threads in various forums where some of you are very skeptical about measuring productivity, effectiveness, and efficiency metrics of testers.

Some other proclaimed gurus and testing experts are completely against metrics. In order to form an opinion on metrics one should first try to understand what benefits one can get from metrics and KPIs and why do we need to measure anyone’s productivity.

The big misconception is that productivity metrics are not fair as they can paint a completely inaccurate picture and are used only as a tool to identify poor performance or used against employees when it comes to firing decisions or salary revisions and appraisals.

I would agree that improper collection of metrics can lead to inaccurate outcomes and as a result, unfair decisions. Therefore, instead of refusing and blaming metrics for usefulness and unfairness one should understand how to collect metrics the proper way and how to make metrics collection and calculation a fair process which will provide benefits to all.

One thing I realized throughout my 25 years of career in Software Quality Assurance and Testing individuals who do not like metrics or biggest opposers to metrics simply afraid of them.

They are afraid to measure their own productivity within organization and value add.

They are afraid to be revealed that they are not effective because they only can talk fluff but cannot execute or take actions.

Each employee should bring ROI to the organization they work for. How is this ROI measured? Yes, you guessed it right, it is measured through productivity and effectiveness.

Below are my favorite metrics which help me to understand better on what throughput we as organization QA Mentor - Software Testing Expert - 14 Years of Making Products Successful can achieve for our clients and customers, provide better insights on test estimation, timelines, capabilities, future projections and value add from each QA Professional.

?

Test Effectiveness of a Tester:

• Percentage (%) of Defect Leakage against assigned User Story
• Percentage (%) of Defect Leakage against assigned Functionality
• Percentage (%) of Defect Leakage against assigned Module
• Percentage (%) of Defect Leakage against assigned Regression Testing Cycle
• Percentage (%) of Schedule Variance for a Tester
• Percentage (%) of Effort Variance for a Tester
• Percentage (%) of Defect Rejection Rate
• Number of Suggestions/Recommendations/Improvements raised by a Tester
• Percentage (%) of Suggestions/Recommendations/Improvements raised by a Tester
• Number of Critical Defects identified per Sprint/Release/Week/Month/Year
• Number of Static Testing defects identified per Sprint/Release/Week/Month/Year

?

Test Case Design Productivity Metrics:

• Number of Test Cases Designed per day (UI)
• Number of Test Cases Designed per day (API)
• Number of Test Cases Designed per day (Functional-Positive)
• Number of Test Cases Designed per day (Functional-Negative)
• Number of Test Cases Designed per day (Integration)
• Number of Test Cases Designed per day (Database/back-end)
• Number of Test Cases Designed per day (Field Validation/Boundary Condition/Error Messages)

Test Case Execution Productivity Metrics:

• Number of Test Cases Executed per day (UI)
• Number of Test Cases Executed per day (API)
• Number of Test Cases Executed per day (Functional-Positive)
• Number of Test Cases Executed per day (Functional-Negative)
• Number of Test Cases Executed per day (Integration)
• Number of Test Cases Executed per day (Database/back-end)
• Number of Test Cases Executed per day (Field Validation/Boundary Condition/Error Messages)

Watch this video on how to measure TEI (Test Efficiency Indicator) and Defect Leakage the proper way -> https://www.youtube.com/watch?v=C33niq92t7A

Your comments, feedback, concerns are always welcomed.