Testing Metrics: Are you measuring it the right way?

Often, managers present testing effort data as a measurement of Testing progress. Though it is one way of measuring the progress in the software testing cycle, it gives you a little indication of testing effectiveness. Both sets of metrics are needed to ensure whether the testing objectives are met or not. It also helps in identifying if there is any tweaking in the test strategy.

Let's see what can help us measure testing effort:

1. Test case count: Number of test cases that have been created and executed as part of the testing process
2. Testing time: Amount of time that was spent on testing the software or particular user story
3. Testing resources: It has multiple sub-categories like the number of testers, the number of test environments and the types of testing tools.
4. Defect count: Number of defects per user story/module/functionality
5. Defect resolution time: Amount of time taken to fix a defect. This data can be further accumulated to represent a functional module.

Now, let's look at some of the ways to measure testing effectiveness:

• Defect density: Number of defects found in the software component / Module by the size of the software (size can be represented as the number of lines of code)

Defect Density = Average number of Defects/KLOC?

*KLOC - Line of codes per thousands

It can help you focus testing in the areas that represent higher defect density values.

• Test coverage: Percentage of the code tested/covered divided by total lines of code. It is a good indicator of understanding actual test coverage.

It can also be represented as how much % of the requirements are covered by the test cases.

• Defect severity: This defines the impact on the software by flawed functionality instead of the effort needed to fix the defect. A higher number of high-severity defects indicate that particular functionality is more vulnerable to late-stage defects.
• Defect resolution rate: Percentage of defects that have been successfully fixed over a fixed period of time. A higher defect resolution rate indicates that the team is effectively addressing and fixing defects in the software.
• Defect detection time ratio: Amount of time that it takes for defects to be identified in the software.