Defect Leakage - a silent destroyer of your product

What is a defect leakage?

The basic answer is that it’s a malfunction that slipped past the whole squad as well as any test scripts in place, ending up in production and potentially affecting users..

Defect Leakage is a metric used to determine the performance of QA testing, specifically how many defects are overlooked during QA testing.

Defect Leakage = (No. of Defects found in UAT / No. of Defects found in QA testing.)

There are numerous reasons that result in bugs in the production environment. A large number of bugs will eventually bring a bad reputation to the software teams as well as the organization. A sour relationship with the client may also result, and one may lose confidence in providing a bug-free application. It will also lead to extra effort and stress for the development and testing teams.

Possible reasons for Defect Leakages and possible ways to to avoid them:

1. Typically, development teams will perform unit testing on the Dev environment, functional teams test environment, and the code will then be moved to business testing (UAT). Most of the time, these environments differ for a number of reasons such as security, performance, roles, and so on, as the UAT is a staging environment that is similar to a production environment. — — — To avoid these deviations, teams should conduct thorough unit testing and try to obtain at least one production user access request from the end user to perform end-to-end testing before moving it to UAT and then to production or inform the end user beforehand by documenting with all the limitations in terms of assumptions and risks.

2. No significance to testing. It has also been observed that projects and organizations place a high value on testing on paper or during meetings only. In practice, however, they try to minimize the testing effort by delivering the developed application at the last minute, allocating limited resources, or providing less time for testing team. To avoid all these, projects should implement shift-left approach by involving testing teams early in the projects, should also involve them to frequently interact with developers and help them in creating unit testing scenarios. Any dependencies must be resolved at the beginning, and the developers must understand the user stories/requirements thoroughly. To clear any kind of clarifications, regular touch base meetings, one-on-one sessions, and daily calls should be used. This way the development will be completed quickly and testing team will have a breather space to perform the testing.

3. No business knowledge to testers : It is not enough to understand the requirements and believe that by writing a test scenario, the testing team can perform better testing. To fully comprehend the application, it is always advisable to have good knowledge on the business side of operations, acquire domain knowledge, and have regular interaction of QA teams with business people***. Management should strive to facilitate certain training sessions with the business testers so that dev and QA teams will have better knowledge of the domain.

4. Irresponsible QA teams : QA teams are quite often negligent during functional testing, assuming that end users will do the validation anyway because they are aware of the entire scenario. This type of behavior could have serious consequences for the application. Testers should have a positive mindset as well as a positive psychological environment in order to perform testing with the utmost care, and they should always take an interest in exploratory testing as well

5. Less test coverage : Teams lack adequate test suites that cover the entire scenario, whether it is unit testing by development teams or functional testing by the QA team. Teams should have a larger scope and variety of test cases, as well as test steps, by using a good test approach for load testing, performance testing, and security testing. Run the regular smoke and sanity tests to automate the test suites wherever possible. Make a bound script of test cases and scenarios for each requirement.

6. Psychological safety : The most important aspect is to have good relationships between testers and developers. Furthermore, healthy interaction between these two teams will result in good psychological safety. The QA team should not be afraid to approach the development team to clarify any code-related issues. Similarly, the development team should always be more friendly and willing to explain and clear things up. Eventually, all of the teams are working toward a better, bug-free application and customer satisfaction. One should not consider getting bugs to be a negative remark that will harm one’s performance. Getting bugs and catching them early on is a good way to build great applications.

There are many other factors that contribute to defect leakage and the actions that can be taken to reduce them in software development, but I have only mentioned a few of them based on my practical experience. I would like to encourage readers of this article to provide feedback and to share their experiences in the comments. After all, we need a bug- and guile-free application.