Does Automation = Agile Testing?

Often I hear that the Agile testing and test automation is one and the same. In my experience test automation plays an important role in an Agile testing strategy surely, but the ultimate goal has nothing to do with automation, exploratory testing, or any other QA practice. The goal of Agile testing is to make all known levels testing an integrated, natural part of the Agile development process used, be it Scrum, Kanban, ScrumBan, Lean or any other Agile framework.

To me, the most important aspects of Agile are those of communication and collaboration; cross-functional teams together working closely, preferring interaction to documentation. To succeed, quality has to be an integrated part of this process. Engineers that with a QA mindset are essential participants in the very first discussions, at sprint planning meetings and at the daily standups. They continuously scrutinize the product to assess quality levels and issues testing activities proactively, be it with UX designers (usability testing), developers (code reviews, TDD/BDD, unit-testing) and operations (load and security testing).

The Agile testing quadrant created Lisa Crispin gives a good overview of common testing practices and how they come together in an Agile project:

Moving from left to right, testing activities go from supporting development (i.e., checking that stuff works) to critiquing the product (i.e., trying to break stuff). From bottom to top, tests are technology-facing (i.e. testing at the technology level) to business-facing (i.e. testing from the user perspective). Although none of these testing practices name in the quadrants are specifically "Agile", it is when you apply them in an Agile project integrating the practices as part of your Definition of Done for all backlog items.

Now let's add to this the concept of test automation which is the use of special software to control the execution of tests and the comparison of actual outcomes to predicted outcomes. The primary goal of test automation is usually to run a potentially large sets of tests repeatedly to make sure software doesn’t break because of introduced changes. Since most Agile development teams adopt the practice of Continuous Integration, automated tests are executed as part of the continuous integration build process routinely.

As you can see in the above set of quadrants, automation could definitely cover those QA/testing practices that are technology-facing and that support the development team. Automation extends to some degree upwards to business-facing tests then, for example automated UI Tests and rightwards - to tests that are targeted at finding “holes” in the product - such as automated load tests and production monitors. Automation definitely doesn’t make sense where human expertise, intelligence, and assessments are key manual exploratory testing therefore, usability testing and overall acceptance testing still needed and require “out-of-the-box” thinking to provoke the product and identify inconsistencies and irregularities that are impossible to code for in an automated test.

For my the bottom line is the goal of Agile testing is to make all known levels testing an integrated, natural part of the Agile development process. Automation plays a key role to test more efficiently and within every single Sprint which greatly shortens the feedback loop in the development process. I frequently recommend to teams to include test automation of each acceptance criteria listed in the backlog item as part of their Definition of Done. This just gives a good start in your integration of Agile Testing journey.

All views are my own.
By Jim Saliba