Optimizing regression testing in Agile development
If in your Agile project, you find yourself torn between ensuring regression testing coverage and sticking to a release deadline without the possibility to have both, read below article having best practices and solve your regression testing dilemma.
Regression testing often becomes a pain point for an Agile project manager, as it presents two major challenges:
- Increased time and efforts. Regression testing may take up a lot of time, especially in large projects. Sometimes, teams have to spend an entire sprint on regression, which is hardly acceptable. And increased time and efforts mean increased costs.
- Lack of test engineers’ concentration. Running same test cases again and again for a long time results in the lack of concentration. This can lead to bugs sneaking into production, which also increases overall project costs.
Regression testing in Agile: How to optimize?
To streamline regression testing in Agile development, test teams to leverage the following methods: combining iteration and full regression testing, prioritizing tests according to a risk-based or a collaborative approach, and automating regression testing for increased velocity required by Agile.
Let’s have a closer look at the pros and cons of each method.
Two-level approach
Following this approach, a testing team breaks regression testing into two cycles:
1. Iteration regression. The team performs iteration regression at the end of each sprint. Iteration regression specifically focuses on features and changes made in the iteration and areas of the application that could have been affected.
2. Full regression. Test engineers run full regression before a release to ensure that the application works as planned.
However, the two-level approach is not as simple as it seems. To be successful, this approach requires a seamless communication within the project team. Communicating with developers, the testing team will be able to get a comprehensive view of what was done in the iteration and make a well-grounded choice about the necessary type of regression testing.
Test prioritization: Risk-based approach
Applying the risk-based approach to regression testing, a testing team chooses test cases covering the application areas affected by recent changes in the project and rank them according to the priority, which reduces time and efforts.
- High priority. In the majority of our projects, these test cases made around 10% of all regression test cases. They cover the critical functions of the software, software areas highly visible to users, defect-prone areas, and areas that underwent many changes.
- Medium priority. These regression test cases (about 30%) describe exceptional conditions (negative test cases, boundary value test cases, etc.). This priority mostly applies to the test cases that repeatedly detected bugs in previous product releases.
High and medium priority test cases are always included in the new agile regression test suite.
- Low priority. These test cases (the remaining 60%) cover the rest of functionality. Test engineers use them in regression testing before a major release to ensure full test coverage.
Thus, assigning priorities helps reduce time and efforts on regression testing in Agile with no damage to product quality. However, this approach may cause some problems, as the testing team is limited to their own tasks and cannot see the whole picture. Luckily, there is another strategy that fully correlates with the Agile specifics.
Test prioritization: Collaborative approach
This approach allows all project team members to contribute their knowledge and unique perspective to regression testing. During a sprint, a team works out a covering every change that was introduced during the sprint. The testing team arranges the cards according to what should be tested first. Other team members then suggest corrections to the proposed order. For instance, a developer understands that a recently added feature may influence critical functionality, which was not obvious for the testing team. When it’s time to run regression testing, the team consults the board and assesses time investments.
Though reasonable and convenient, prioritization approaches need caution, as they may lead to technical debt.
So how is it possible to optimize Agile regression testing and at the same time control technical debt? The answer is in regression testing automation, the third optimization strategy.
Regression testing automation
In our projects, automating regression testing in an Agile environment has proved to be the best way to deal with most common challenges this type of testing presents. For example, for one of our clients, we have opted for test automation to optimize regression testing of a one banking product application. Automated regression testing allowed us to reduce the testing time and spared a testing engineer the tedious task of running the same test cases again and again.
The right time is the first thing to consider when planning regression automation. Manual efforts are still required at the early project stages, when software undergoes changes in the UI, product flow or logic. We advise to introduce automated regression testing when the project has already reached some level of maturity, and major changes have already been made. Besides, we recommend verifying automated regression results with quick manual smoke tests to identify false positives/negatives.
Besides, automated regression test suite, however comprehensive, cannot be valid forever. In Agile, automated regression testing demands that the test suite fully reflects the changes and updates in the application functionality. Therefore, review it after each major release and discard obsolete test cases that no longer contribute to the product quality assurance.
Focus on communication
To optimize agile regression testing, a testing team should ensure seamless communication with other project team members:
- BAs to monitor the changes in requirements and assess them from the testing perspective.
- Developers to learn which changes they made during a particular iteration.
- SMs/POs to update them on the current state of testing and avoid time crunches at the end of a project iteration.
Enjoy quality at speed with truly agile regression testing
Regression testing in Agile is like a bitter pill: unpleasant but necessary to assure the product quality.