What, when and how to test
The next pivot for Continuous Testing @ Scale and Speed - What, when and how to test
We all know that the cost of identifying (and fixing) a defect late in the lifecycle. The same applies for speed as well! It is much faster to execute scripts in the earlier stages than later in the cycle. We would be able to run hundreds of tests covering thousands of lines of code in a few minutes if executed at unit level. This goes down as the code progresses in the lifecycle to system or system integration or UI layer.
The speed of test execution, coverage of code and discovery of defects are influenced by what is tested where. It may be efficient to validate edge conditions and crashes at unit level, system behavior for various data combination at API/ services layer than to test the same using the UI layer. Similarly, the objective of testing should also be nailed based on where the test is being run. For instance, at UI layer teams need to validate the layout, display and other cosmetic issues rather than functionality.
Image Courtesy: https://abstracta.us
For test automation at speed and scale, the testing triangle has to be inverted with maximum test coverage at the unit level, and is limited to acceptance test at UI level. The purpose of testing at different levels also has to be clearly identified so that tests are efficient and are able to unearth maximum defects through automation.
See you shortly with the next and final version of Continuous testing @ Speed and Scale