Continuous test improvements in the age of DevOps

Introduction

Lately there is a huge traction towards DevOps in the IT world. Many companies have adopted agile processes in pursuit of faster releases. So the logical next step to this is DevOps. Now every other organization wants to jump onto the DevOps band wagon. If we study the DevOps landscape, testing is a critical part of the handoff between development and IT operations. So testing should be continually improved for the success of agile teams using DevOps practices.

Continuous testing in DevOps

Every one of us who understand DevOps are familiar with Continuous Integration and Continuous Delivery. But the most important bridge that keeps these two entities to work effectively and which makes the concept of DevOps successful is ‘Continuous Testing’. It is not just automation as most people think. Continuous Testing goes beyond automation and encompasses all practices — including tooling and cultural change. It is mechanism of continuous validation that keeps the DevOps engine moving forward. It provides continuous feedback to the project teams to guide them towards successful software implementation.

Continuous testing in the context of DevOps should take a test-first approach, which will help measure test coverage to check that software features work as expected. It will enable to find if changes break existing test suites. And also help the project teams to focus on what they need to deliver.

As part of continuous testing, test teams can also benefit with ‘Exploratory Testing’ by simultaneously understanding the system, designing the tests and executing the tests. This will add value to the test automation by actively exploring the system without a script to learn how the system behaves in unanticipated ways.

Continuous test improvements

Test improvements are a critical part of continuous testing. They drive the continuous effort to optimize test performance of the software. This should and will provide improvements through automation and reuse of best practices. The continuous test improvement can set thresholds for each test phase that determine whether  failures  are significant enough to revert the changes or it is necessary to stop the DevOps cycles long enough to fix the problems. Test first driven development, behavior driven development concepts allow test processes to improve in a DevOps Model. Some of the key areas that can be addressed as part of continuous test improvements are code analysis and test optimization & virtualization.

While test automation acts as a driver for detecting software defects, the concept of continuous testing highlights process improvements that can prevent future defects from occurring.   Test improvements enhance real-time assessments and continuous measurements to refine the development process so that business expectations are continuously met.   

Conclusion

Continuous testing is a great complement to the DevOps movement. Organizations cannot be successful with DevOps if they cannot improve the existing processes. They should know how and what to improve. Continuous testing which acts as an enabler to test improvements can be a good medium through which success is achieved. What is more important is that the ideas of test improvement should be embedded into the DevOps culture.  As I said in my earlier posts, quality is built into the entire system with DevOps. When implemented the right way, the concept of continuous testing in DevOps will be more than a method of enforcing test automation tools or best practices.