Can we do test automation , the right way
Satyendra Thakur
SDE 2 @ Amazon Music | Performance Test Automation, Software Development
After working on automation QA intensively for more than 5 years, I am grateful to each and every opportunity that has helped me grow as a professional and as an individual too. This is one of my baby steps to give something back to the community.
I hope you can make some sense out of it.
Disclaimer ! My writing sucks though .
What is automation QA ?
In software testing, test automation is the use of software separate from the software being tested to control the execution of tests and the comparison of actual outcomes with predicted outcomes.
This is how wikipedia describes it , but is it really how it is marketed ?
Expectation vs Reality
Through automation , we can finish testing faster and more accurately because it helps save time and trims redundant task . This really sounds cool and this is how we sell it to clients too. However, this is the expectation but I am really afraid to admit that the reality lies at the negative infinity.
It's true that automation can reduce redundant tasks, if only it doesn’t become a redundant task in itself. In the quest for solving problem , I ‘d rather say , we have redesigned the problem instead.
A typical process of automation , what has been set as a practice in many businesses is :
- Choosing the right tool ( “right” word is a myth here , at many occasions, people are left with the open source tools as the only choice )
- Customise the tool to meet the requirements , which we call as framework development . It can be BDD, Hybrid , data-driven however you want to name it.
- Develop test suites in (n-1) sprint ( one sprint behind or sometimes in parallel with the product development )
- Automate all business requirements which the automation team thinks is feasible to do. And leave behind the ones that are not feasible to automate.
- Create an automation execution plan and try to justify the plan to business
- Wait for the stable build and run the test suite
In following this process , automation QA typically loses its gists and we run in an unending loop to update test suites everyday by changing the codes to make it ready for running it on some dedicated day , the grand finale !
But it is important to recall here that , The money invested in automation is a lot more than on functional QAs. But to go by statistics , it's the functional QA that finds more number of issues and do some really structured and haphazard exploratory testing that helps in finding even show stoppers and critical flaws in the application. Then what are we really spending money on ?
Where do we tend to go wrong ?
If we can trace steps back to the beginning , we would be surprised to know that we ourselves are the culprits and we become the victims later.
- The theory that is sold to client that " Test automation can reduce days of tasks plus can help in reducing team size " is something which sets the wrong expectation in the mind of product owners. This theory in itself is so fascinating that nobody really tries to dig deeper and dares to ask back , " On what occasions will test automation reduce days of tasks ? What are the possible challenges that we might face ? And most importantly , can this beat human intelligence in doing QA ? "
Can we do it this way ?
- I believe it's the right time to blow out dusts and cleanse the theory of test automation.
When we have a common team who is set to do all round testing of the product , through human intelligence plus making use of their coding skills in solving the problems which otherwise would take unnecessary amount of effort , the problems starts solving by itself.
I believe if only we can stop discriminating between a Manual QA and an Automation QA and promote the merge of these two practices as Intelligent QAs then we would actually be able to revolutionise the testing process and make the best use of testing through code and testing with Human Intelligence .
Now If we can go back to the answer of ” What is automation QA ? “ and try to stick with it, we can actually go miles in optimising the testing effort.
Happy Learning !
Data & Interoperability| Solution Architecture | Data Architecture | Cloud | Healthcare Data Standards
4 年Good read. I do agree that in the race of 'Automation first' and 'save efforts' philosophy, we are actually overlooking one of the very key aspects of feasibility/due diligence to determine where the automation would fit in and if it will yield the ROI as intended.
Research, content, and code for cryptonative public goods
4 年This! ?"...if?only we can stop discriminating between a Manual QA and an Automation QA and promote the merge of these two practices as Intelligent QAs then we would actually be able to revolutionise the testing process and make the best use of testing through code and testing with Human Intelligence ."
Manager at Deloitte Digital
4 年Very brave attempt Satyendra Thakur. I have not seen many automation experts taking this topic so seriously. All these years i have always heard, yes this can be automated. This can be done in Sprint (N-1) or parallel to development. But there is a rare breed of experts thinks about making automation successful. Successful by making it profitable during the product development phase itself. There are talks about ROI from automation but as rightly said by you that the success rate for this is very low as we always run behind making automation scripts workable and demo-able to client. I strongly feel that it is time to say that we will work for quick wins first - like help manual QAs doing a well planned exploratory testing by providing them large amount of data super quickly, by creating some scripts which can run few scenario for basic sanity rather than completing all US (if agile is followed), finding ways to test accessibility test automatically after each build, analyse result and report back, create tests for mundane work. Test APIs more frequently etc. There are of course cases where automation can be done in parallel or N-1 sprint plan may be successful and useful, but it is our responsibility to make sure that we do a careful analysis before jumping for completing automation as soon we hear the word GO.