When testing and development becomes a wild goose chase

In what Forrester Research call the “Age of the Customer”[1], markets are driven by the changing needs of customers who demand quality and speed like never before. Organizations must be able to deliver software which reflects these changing requirements, or be forced out of the market by a competitor who can. This is true whatever industry a company is in, as recent market leaders in consumer electronics, transportation, and hospitality reflect.[2]

As a consequence, more and more development work is now driven by a series of emails and written change requests, rather than the complete specification which stood at the start of a Waterfall project. Test teams are now under pressure to react to change quicker than ever before, and yet change remains a frequent cause of project delays. This is because many of the bottlenecks which arose in traditional environments remain, even though the way in which software is designed has changed.

For example, change requests are stored in the same “static” formats as traditional requirements, such as written documents and flat process diagrams. Testers cannot therefore easily identify the impact a change will have on a system’s logic, or how it will affect the complexity of a system. An apparently “simple” change like adding a new decision might cause a system to fall down, while a relatively low-priority improvement might end up requiring months of work.

Manually updating tests each time a change is made further causes project delays and budget over-runs. Without being able to identify the impact of a change, every existing test case has to be checked and updated by hand. This is slow and laborious, and at one organization we worked with it took 7.5 hours to check and update the relevant test cases by hand each time a change was made to a single ETL transformation rule.

Often every test case is “burnt” and written from scratch, whileanother tactic is to pile up more and more tests in an effort to maintain coverage. However, this leads to “test case overload”, where broken, duplicate and redundant tests never die. Over-testing is rife, while invalid tests fail when there is no genuine code defect. The failed test case then has to be investigated, while developers might be sent on a frustrating wild goose chase.

Test cases and system requirements clearly need to become more reactive to change if organizations want to remain competitive. One way to achieve this is to use model-based testing, where the static requirements are replaced with an “active” model like a flowchart.

Because a flowchart is a mathematically accurate model of a system’s logic, testers can use automated algorithms to identify the impact a change will have on the system. This is true for individual components, as well as system-wide: when a change is made to a component (a sub flow), its impact on higher level, master flows can be automatically identified. Testers can therefore know exactly which components need to be re-tested to make sure a change has successfully “rippled up”, and can measure how complex implementing a change will be.

The time wasted manually updating tests is also eliminated with model-based testing. When a change is made to test cases on a lower level, with a flowchart tool like Agile Designer, the impact this will have on test cases stored in the relevant master flows will be automatically picked up. Test cases can then be automatically updated system-wide, creating any new tests needed to fully test a system, while removing any redundant or invalid tests.

This means that testers only re-run the tests required to validate a change, and no longer need to manually check and update test cases by hand. Testing therefore becomes fully re-active to changing business requirements, so that organizations can deliver software fast enough to reflect changing user needs, and with the quality required to satisfy them. An organization’s place in the market is therefore secured against new and innovative competition, while also allowing them to innovate and lead markets.

[1] https://www.youtube.com/watch?v=BAbqNoxvdxM

[2] https://fortune.com/unicorns/

[Image: Pixabay]