Test Automation Principles
Minh Hoang
?? Head of Quality Engineer at GiaoHangNhanh (GHN) | ?? Quality Transformation | ?? Foster Quality Culture | ?? Provide Testing Solution
When we talk about Test Automation, many people will think it will help the development team with the objectives below:
However, during my career as a tester, I have observed many projects that can't maximize the benefit of Test Automation. Most of the projects had a problem when they did not have a good guide for behavior or evaluation when working with Test Automation. From my experience, I always use the below Test Automation principle when working with it to maximize the benefits it can bring to our team.
I follow simple principles, with a one-word TEST, a mnemonic acronym for four principles. It is intended to make Test Automation increase test efficiency and effectiveness and shorten the time to provide feedback to a development team.
Think about test design and test it first
First of all, Test Automation does not mean we have a machine that can perform testing for us, it does not like we have a magical device, and when we press the button, it will provide us with the risk information about what we don't know. You can ask yourself the questions below "who will create an automation script?", "What do we create an automation test script based on?", "If we have a new feature, will the automation script can automate testing for us?". I hope those questions can help you have another view on Test Automation; it is like how we have many different labels in our job, such as QA, QC, and SDET. Still, we all know our job is to instill confidence in delivered software through various testing activities.
Testing is the activity that needs humans to explore and gather information about the system. And if we don't have a good test design, then the Test Automation can't bring any value, and we can't expect Test Automation to cover any case that humans are unaware of. As a tester, you need to think about how you will approach a testing problem, and you need to test it first and then figure out what's suitable for automation.
Extra tool to support test
Test automation is just a method of using different tools to help you in testing activities. Nowadays, we have many tools to support your testing, such as automating the GUI; we have many tools like Cypress and Playwright besides Selenium; for gRPC testing, we can use BloomRPC or Kreya. We have many choices for selecting the tools to help us in testing, and test automation is just a way we think to find a means to support our testing mission.
During my testing journey, I have had the opportunity to use and contribute to many tools that can support testing; thinking about test automation, I consider how I can leverage the power of a computer to assist me in my testing activities.
领英推荐
Speed, stability, and reliability in your Test Automation
Without speed, stability, and reliability, test automation becomes flakiness. Flaky tests hinder development, slow-down progress, hide design problems, and can cost money in the long run.
Speed: test automation needs to be executed as quickly as possible. Currently, for CI/CD, it is impossible to integrate with the test automation, which runs for 8 hours, even 3-4 hours, it is too long, and we can't provide a fast feedback loop to the development team.?
So for you to always have a fast feedback loop with test automation, you can try the FIRST principle. Also, you always need to see if your execution time is optimized. Is there a way to reduce the execution time (split multiple sets, run various groups in parallel, etc.)
Stability: test automation needs to run consistently. If you are doing automation, you must have heard the term flaky test (this is a term to refer to test automation scripts that suffer from run-pass and run-fail conditions). Imagine if you have automation in your project and where most of the test scripts are suffering from flakiness ?? At this point, you obviously will not feel confident about the results that automation brings. So to help test automation reduce flaky test scripts, you can refer to the following links:
Reliability: test automation needs to have the highest reliability; reliability here means that the test automation not only has a low rate of flaky tests but also ensures the correct verification of the business logic that the test is executing. I have encountered many cases where a test script has many actions inside, but I can't find a place to verify the business logic of that test script; it means the test does not have an assert action or the assertion is incorrect. So to ensure reliability, you should use the break-test technique, where you will inject failed data into a system under test and use your test script to check if the test script fails when to run, to double-check the reliability of the test script.
To automate or not to automate
This principle emphasizes that not everything can be automated; as I know, if I aim to automate every test case, it will make my test automation very cumbersome. As a result, test execution will be slow, analysis and maintenance time will be more, and the benefits of test automation will decrease over time, and you may spend a large amount of effort instead. To maintain and analyze it. So how do you know when to automate and when not to? Ask the questions "Is the current system ok to automate?", "Have I tested that feature enough?" To automate, you can use Google's ACC model (Attributes-Components- Capabilities) to generate a risk map. From there, you can focus on knowing which parts to automate and which parts should not be automated.
Summary
This is my principle when working with test automation, and willing to receive your thought about your guideline when working with test automation :D