Let's Be Reasonable: QA Automation Will Not Replace Manual Testers Any Time Soon

For a long time, I've been told that QA automation will soon replace manual testers.

This view is rooted in we-can-automate-it-all philosophy. 

That philosophy is wrong and harmful to both the software industry and QA professionals, and it often results in creating test automation for the sake of test automation. 

Let me illustrate: 

1. What happens if we try to implement the we-can-automate-it-all approach in real life 

and 

2. Why manual testing is and will remain a valuable asset to the software companies. 

We’ll focus on the main application of test automation — automation of functional and regression testing.

—

Let me share a personal story. Many years ago, I joined a Silicon Valley startup as its first QA engineer. The product was a web-based service. 

I was hired right after the company had released v1.0 to the public and the customers started to write angry emails that some things didn’t work as expected. 

The first task I received from our then-CTO was: “Write test automation for our site. ASAP!” 

Here is our dialog:

R: OK, let’s talk about it. Why did you hire me?

CTO: To make sure that we release our software without bugs.

R: But why do you think we have to do automation first?

CTO: Because when we automate it all, our testing will take minutes.

R: I agree; the automated test execution runs much faster than manual testing. By the way, I looked at the checkout flow today, and I saw some UI changes compared to yesterday.

CTO: True, we change UI all the time.

R: Just like app logic and DB schema?

CTO: Yes, we are a startup and have to move fast.

R: Now imagine that instead of 

   - creating checklists for acceptance testing

   - setting up the bug tracking system

   - auditing the SDLC

   - writing test cases for functional/regression testing

   - executing those tests manually

   - writing API-based helper tools for repetitive tasks, like account creation…

… in other words, instead of implementing QA/Testing ESSENTIALS, I'll focus on writing automated tests for a checkout flow. Technically, it’s not difficult, but my automated tests will be linked to UI that we change every week to be conservative. It means that every time UI is somehow altered, e.g., a new text field is added or removed, or we introduce a new logic, I need to change my automation scripts too.

CTO: But is that so hard to change your automation?

R: It’s not hard when we talk about one flow and proper test automation architecture. But we are talking about tens of flows that keep changing on all levels - UI, App Server and DB. The outcome is predictable — I’ll be spending my days on the maintenance of test automation scripts because they’ll be breaking whenever linked part of the software is changed. When you ask me to run my scripts before the release, only a part of them will run without errors, and the other part will always be broken. So I have to do manual testing in any case, or we have to hire manual QA to execute test cases that are no longer automated.

—

As a result, we decided to postpone test automation and concentrate on process improvement (QA), setting up test infrastructure and manual QA with grey box approach. We did hire automation tester in a while and fired him in several months because his scripts were broken all the time — our startup software was dynamic, and our PMs were full of new ideas.

—

The main strategic mistake towards QA automation is focusing on HOW to automate instead of WHAT to automate.

HOW to automate is about tools and test automation architecture.

WHAT to automate is about finding a piece of software suitable for automation.

HOW to automate is a technical aspect.

WHAT to automate is QA aspect.

If we consider HOW to automate, we hire a software engineer who knows how to program interaction with software via automated script using the test automation framework.

If we consider WHAT to automate we look at ROI, proper balance between manual and automated testing, release schedule, esprit de corps inside QA department, etc., etc., — essential things for the survival of our business. 

Selecting WHAT to automate is a crucial FIRST step before investing a dime into QA automation because otherwise, we'll waste our resources by creating QA automation for the sake of QA automation.

—

If we think about WHAT to automate, we’ll see that only a limited number of candidates are suitable for test automation. Namely:

1. STABLE components

2. STABLE end-to-end flows

3. Repetitive or labor-intensive manual tasks

1. and 2. Stability/maturity of software means that it’s not subject to frequent changes. It’s also logical that a single (more so standalone) component would have less probability of change than the end-to-end flow that includes many components. But as time goes by, SOME end-to-end flows would mature too.

Hence, if we should test that STABLE software frequently, we can consider automating its testing, because

a. We don’t have to spend resources on constant maintenance

b. We free up the time of manual testers so they can concentrate on 

i. Manual testing of the NEW functionalities 

or  

ii. Manual testing of things that cannot be automated (e.g., user experience) 

or

iii. Better coverage of existing core flows.

3. If you think about QA automation as a career, I can give you excellent advice: consider writing HELPERS as an essential part of your work. 

A helper is a tool that automates some part of manual QA or dev work. 

For example, during my PayPal times, we had to test integration with some banking institution. In the process, we had to create a settlement file in the specified format with data retrieved from our DB. It was a problematic and repetitive task that was also very important for our business and prone to human error if done manually. So, I’ve created a Python tool that would 

- generate a credit card transaction,

- retrieve data from DB and create a settlement file,

- push it to the test servers of the bank

- read the response

- and give Pass or Fail + response text to a tester who ran the tool. 

What used to take an hour was now taking two minutes. The automation was a success because it satisfied the need and used stable components. It was automation for the sake of the company, not for the sake of automation.

Helper tools usually include:

a. Automation of repetitive tasks, e.g., account creation.

b. Data/transaction generation needed for testing or development.

—

I intentionally covered conceptual things about proper test automation to demonstrate that test automation alone cannot achieve the level of quality that a mixed team of manual and automation testers can.

As a team, manual testers and automation testers:

1. Combine their powers 

and 

2. By using different approaches to test software, they neutralize the weaknesses of another party.

—

And lastly, let me give you another compelling argument in defense of a value that manual QAs bring to their company: if you have an opportunity, log into your bug tracking system and see how many valid bugs were filed by manual testers and how many by the automated scripts. 

In most cases, you’ll see that manual testers log tens or even hundreds of TIMES more bugs even if a headcount of automation testers equals or exceeds the headcount of manual QAs.

—

Thanks for reading, likes, shares, and comments.

And BTW, if you want to build a solid QA organization within your company, consider buying our course How to Become a QA Tester in 30 Days for yourself and your employees. URL: https://elearning.qamentor.com/p/how-to-become-a-software-tester-in-30-days