Goal-Driven KPI Approach in Software Quality Assurance

"Count what is countable, measure what is measurable and what is not measurable, make measurable."

Galileo

The following are a number of useful Key Performance Indicators (KPI) used for many organizations in the Software Quality Assurance programs:

• Time to Fix Defects - number of hours from the time defect was found until it was resolved
• Defect Removal Efficiency (DRE) - number of defects in test phase / (number of defects in test phase + number of defects in production until 30 days)
• Test Case Effectiveness (TCE) - number of defects not mapped to test cases / number of defects mapped to test cases
• Traceability / Code Coverage - % of test cases that map directly back to requirements and % of requirements that map directly to test cases
• Planned versus Actual Execution - line chart test cases planned versus executed daily
• Defects Distribution by Severity/Status - bar chart defects by Severity (by project/test cycle/etc...) or bar chart defects by Status (by project/test cycle/etc...)
• Defects Distribution by Root Cause - bar chart defects by Root Cause (by project/test cycle/etc...)
• Testing Effort - actual testing effort (Full Time Equivalent hours) / total overall project effort (Full Time Equivalent hours)

Suggested baselines to start with Key Performance Indicators (KPI) above:

• Time to Fix Defects - 2 hours
• Defect Removal Efficiency (DRE) - 90% defects removed
• Test Case Effectiveness (TCE) - 10% not mapped
• Traceability / Code Coverage - 98%
• Planned versus Actual Execution - 8% deviation +/- from plan
• Defects Distribution by Severity/Status - bar chart actuals
• Defects Distribution by Root Cause - bar chart actuals
• Testing Effort - 25 to 30%