Automated testing – The bottom line why it’s needed early and often

This post was originally published on the MacIsaac Consulting Blog.

You need automated testing early and often because it reduces the cost of defects. “Defect Cost Increase (DCI) is one of the few empirical verified truths about software development: the sooner you find a defect, the cheaper it is to fix”(Beck, 2005). In a traditional waterfall project, all testing happens at the end of the project.

The waterfall based late testing approach results in high defect cost increase. As a former SQA analyst, I have experienced the pain that goes along with this process. Late nights working in WAR rooms, constant stress and continual red status reports. It’s the classic project death march experience.

In Agile and Lean software development, testing happens early and often. The team builds working software in short iterations. Testing is part of the software development process. In the below caption, Figure 14 shows the waterfall style of testing, compared to figure 15 which shows a more agile testing process.

One of the more popular and effective approaches is test driven development (TDD). Using TDD, the developer will first write an automated test. This test will ensure the basic functionality of the code works. After writing the test, he’ll first run it to see that it fails. Then, he’ll write his code, and run the test again to ensure the test passes.

By using TDD, the developer has already taken steps to ensure quality is built up front. The code then goes to QA, where integration, user experience, or performance testing happens. Just as the developer has created his automated test, so should the QA resource.

A high performing QA team will have an automated regression suite ready to run as soon as new code is ready. It is crucial that core functionality regression tests run first, before new feature testing. The idea is, make sure the new code hasn’t broken anything first, then begin the new feature testing. Most organizations have this backwards, doing all the regression testing at the end.

To put this all together you need the right tools, technology and resources. Whether it be JUnit, QTP, or Selenium, find the automation technology that’s right for your organization. Remember, the goal is to reduce defects and their cost.

Many organizations have avoided implementing test automation due to the expense. Ask any software quality team that uses automation and they’ll tell you the ROI is well worth it. If your organization builds software, you must use test automation. By not using it, you’re operating on an unsustainable model. It’s unsustainable because your regression test suite will continue to grow. You cannot sustain a growing regression suite without automation.

For more content like this, subscribe to the MacIsaac Consulting Blog.

To contact us about our services, click here.

References: Beck, K (With Gamma, E.). (2005) Extreme Programming Explained, Upper Saddle River, NJ, Pearson Education, Inc