Book Excerpt - Science of Selenium - Ch-11 Tips & Tricks for Test Automation - AI, Visual, Script Less, Rapid Testing

Dear All

Using this forum to share some excerpts to raise the interest levels of the book I wrote and published through BPB Publications. I will share some excerpts every week to get your inputs and suggestions for future work.

This week, I am focusing on the Chapter 11 Tips & Tricks for Test Automation. This covers the following aspects

Table of Contents

Chapter 11 Tips and Tricks for Test Automation

“Good programmers write code for humans first and computers  next.” – Anonymous

In the 10 chapters of this book, we have covered all the essential elements of Selenium for test automation. In this chapter, we will focus on how to do test automation efficiently. What are some of the good tips to follow for test automation programs? What are some tricks to be followed by the test automation team? This section covers Generic Test Automation Topics. (i.e.) Why test automation instead of manual testing, differences between testing and checking for assurance, automating useless test cases vs designing test cases before automation, removal of uncertainty by having clean test automation, automating stable functionality vs unstable functionality, myths about 100% test automation, partial automation vs. end-to-end automation, inverted pyramid of test automation, 100% test automation vs 100% better benefit through test automation, tools and techniques to leverage, risk-based test automation approaches, etc. We will cover test automation coverage for DevOps and AI-based application development as  well.

Structure

We willlook at the following concepts to test the test automation knowledge and include content as a reference and tips to the following. These include coverage on the following topics:

When & What to Automate?

• What are the differences between Manual Vs Automated Testing? Benefits of Automating Stable Vs Unstable Applications Strategizing right vs automating fast
• Automating all vs automating right
• Record and Hard-coding Vs Intelligent Automation 100% Automation Myth Vs Beneficial Automation

Tips to Automate Better

• Automation Tools and Methods Script-less Test Automation
• Automating the Test Automation Process
• AI in Test Automation
• Visual Regression Automation
• Test Automation for DevOps

Objectives

In this chapter, we willexplore the various aspects of test automation to evaluate what are some good tips to follow, what to automate, when to automate, what is the right approach, what are some suggested approaches,what are the different ways in which we can automate efficiently? This chapter will focus more on the theoretical and concept-based understanding of test  automation.

When and what to automate?

Let us analyze why we need to automate, what we can automate, how we can automate, when to automate, and why to automate and not automate elaborately. Firstly, let us understand why we should automate by comparing the manual and automation testing. We broached on this topic briefly in the first two chapters but this chapter will focus more on an advanced and informed analysis of the   reasons.

Manual Vs Automated Testing – Some comparisons

Let us compare the manual and automated testing in a simple table as follows:

Manual Testing

• Manual Testing need not be consistent. As different testers may follow different approaches and perform actions in an inconsistent manner.
• Repeatability is a challenge
• Manual Testing can be time-consuming
• Parallel  test  execution  would  mandate more resources and more cost
• Investment is on human resources and the effort spent is not reclaimable
• There can be human errors
• Not feasible to run all test cases for a complex application with thousands or millions of test scenarios
• Mundane test execution tasks can be demotivating for talented testers
• Useful when exploratory testing or skilled validation of functionalities are needed
• Customer experience can be a question mark – especially when skilled tester churn happens
• No coding skills needed for testing
• A lot of administrative tasks are required after testing
• If a step is missed or a screenshot is missed out, rework risks increase
• Time and complexity increases based on the number of verification points and test cases

Automated Testing

• Automated testing is driven by logic and automated testing will always be consistent and repeatable in nature.
• The automated script will be behaving in exactly the same way in a consistent manner, every time the script is executed. There will be no change in how a script is executed.
• Automated testing can be used to reduce time.
• We can write once and run multiple times in parallel in different environments at the same time.
• Investment is on the tools and building of the script. The returns will be based on the number of times a script is executed.
• These are consistent and repeatable and there will be noerrors in the process. The script can fail due to many reasons but will not have an error.
• If implemented efficiently, automated testing gives good coverage and the ability to run thousands of test cases in a very short amount of time.
• Automated testing of mundane tasks, especially in regression testing can be done seamlessly using automated testing.
• Not very useful in rapidly changing the UI or deep domain knowledge-centric test scripts that are done efficiently by skilled manual testers.
• Since it is repeatable and person agnostic, the experience is always the same.
• Coding may be required to do test automation.
• All non-value adding admin tasks can be done quickly with automated steps.
• With automated test cases, human errors and rework as a result can be reduced.
• Time and complexity increase issuesthat can be reduced by cross-platform parallel test execution.

Inverting the test automation pyramid

Traditionally, test automation is done at the user interface level when the amount of testing, test automation, cost of testing and time needed are all on the higher side. Almost all organizations fall into this big trap of doing the most important things    at the last moment, resulting in delays, poorer quality, and cost and schedule over runs in addition to getting poor customer feedback. What is meant by inverting the test automation pyramid is to do more automation inthe earlier phases so that the incremental feature validation and automation requirements reduce, as the product quality gets better through the software development life cycle. Let us see this in the following image:

What we are suggesting is to have more automation in theearlier phases such as unit testing where script-less automation or rapid testing is feasible and fixing a defect   in the earlier phases is a lot cheaper than fixing a defect at later stages. Ideally, the test automation at the UI level should be minimal if we do a good job of validating the functionalities, interfaces, APIs, and integration touch points earlier in the SDLC using various test automation methods and frameworks. Use of automation tools to validate a build, validate an action, validate an API, validate an interface, validate     a Micro service, validate data integrity, and validate responses can go a long way. This means we can tiethe system components together in a modular and progressive manner. The ease of automation, effort involved in automation, and the ability to fix the issues are all favorable for an excellent SDLC. The later we start this activity, the longer, inefficient and costlier the entire process will become. The benefit of inverting the pyramid is like flipping the nominator and the denominator and the value could be a lot smaller when the numerator is big when it is flipped. The tautology is very much applicable for the inverted test pyramid – for both testing and test automation.

Test automation strategy

Let us see some tips we can leverage to understand the benefits of automating stable vs unstable applications, strategizing right vs automating fast, and automating all vs automating right in the following sections.

Strategizing right vs automating fast

When it comes to strategizing a test automation project, some of the key questions to ask about test automation before we get started is depicted in the following  table:

In addition to the questions in the preceding table, some additional points to ponder include:

• It is always a good idea to automate primarily around the functional logic and in the zone of implementation of business logic as the checks will be critical here.
• Other checks and balances may be less important and a Pareto’s 80-20 rule may be applicable.
• A tester can consider the risk-based testing approach here to decide what is important for automation and testing.
• Efficient test automation projects will focus on minimizing the impact on the automation framework due to moving parts or by minimizing the impact of end-to-end automation by modularizing  it.
• Hence, an automation framework that is abstract, extensible and easy to maintain is very important to  have.
• Having a mechanism to address the user interface changes easily is very important and most of the maintenance for test automation arises due to the dynamic nature of the user interface  changes.
• With the added complexity of asynchronous JavaScript and other dynamic features, the automation of the UI becomes even more complex.
• This is an area handled very efficiently by excellent test automation tools and frameworks built using Selenium.
• One more focus area will be to address the challenge of unwanted automation or obese automation. To address this, it is wise to ask some of the following questions before we choose the script or test case for automation as shown in the following image:

Figure : Test Automation – Script Selection Approach

Once we have chosen a tool, picked up a framework, and strategized test scripts     to automate, it becomes important to decide what is the purposeful automation objective we can attain.

Benefits of automating a stable application versus an unstable application

One of the challenges faced by automation developers is to decide when to automate? What is the golden period for test automation? Is automating early an option? What about automating progressive and unstable application features and functionalities? Wouldn't manual testing be a more suitable option than automation? Doing automation on an unstable application or a feature will result in a significant maintenance effort of the automation scripts and the benefits we are likely to gain via test automation will go for a toss. This means that we choose an automation window that is likely to help rapid automation with minimal to negligible maintenance effort. How do we do that?

We can consider picking up test cases for automation once the scripts have passed through at least a system test cycle and once the majority of the bugs have been fixed with almost all the features delivered. While there is a risk of bug fixes resulting in code changes and impact to test automation, this ‘Golden Window’ for test automation    is likely to deliver the maximum benefit. This is because the scripts that are likely to be a part of the regression suite in future cycles will be run no matter what the new functionality scope is and will give the ROI needed to make a successful automation project. Hence, the automation priority should be on automating the regression suite test cases during the golden window for test  automation.

In addition to picking up the automation suite, scenario analysis on the impact of failure (business) and the likelihood of failure (technical) of the regression test suite needs to be performed. Based on the priority and ranking, the automation team can focus on the most value-adding regression test cases and finish them first. This is very useful when the team is faced with resource, cost, and timeline  challenges.

It is also important to build various types of regression suites to be executed for various types of testing needs. A core regression pack can be automated for the execution of scenarios for all changes. An extended regression pack (can be further segmented based on the product and functionality features as well) can be used in case of comprehensive regression test needs. Additionally, a smoke test regression suite can be built for the execution of test cases post a build deployment in test environments. The benefit of smoke test regression suite automation is that we can identify build issues immediately, fix them, and not wait for a long time to find issues with the build. Smoke test automation is a very good candidate for getting immediate benefits.

Some of the areas to avoid automation, as it will result in an unstable test automation suite are as follows:

Web pages with a lot of dynamic content and DOM elements: This will result in massive maintenance if not automated well. It is good to have simple objects built in a modular fashion. A page with more than 1000+ objects  will be an automation nightmare.

Web pages and mobile apps with too much asynchronous content: This will result in a lot of asynchronous waits and threads causing the automation scripts to fail. It is advised to go for partial, feature-based automation in-lieu of comprehensive automation.

Pages/Apps with multiple technology interfaces: Assume you are trying to automate an application end-to-end that covers a gamut of technology – for example, a web page that has embedded frames/apps with say Adobe Flash, etc. it technically becomes impossible to automate and handle different behaviors. It is better to avoid them.

Pages with Alerts/Popups/Modal Windows: Unless you know the features and functionalities very well, it is advised to avoid spending too much time on these issues, especially if the alerts/popups and content are  dynamic.

Pages with unpredictable results: Better to avoid automation of these. An example could be a page that is produced dynamically using an AI program.

Pages like CAPTCHA and RECAPTCHA that needs human intelligence: Difficult to pass the Turing test with an automation program and it is better to avoid them.

Other pages to avoid include:

• Native OS applications on a mobile platform
• Pages with multi-level complex workflows
• Apps and content that are single-use
• Test cases for pages/apps with static content (nothing  changes)

Given the current understanding, let us see how can we automate right? Is automation of all test cases the way forward or is it automating intelligently  better?

Automating all vs automating right

As a test automation engineer/architect, it is very important to perform a cost-benefit analysis of the automation, portfolio analysis for test automation selection, script rationalization and selection, and modularization to name some of the essential tasks to be performed by the automation team. If there is a value in doing the test automation, if there is a good team available to support you in the journey and right frameworks designed with simple maintenance tasks in future, you are all set for a good test automation  journey.

It is important to not forget that sometimes partial automation such as micro automation of APIs and Services can deliver maximum value than large-scale end- to-end automation due to the number of uses it would have, including theability to do load testing. Not every automation needs to be end-to-end   automation!

If test automation passes, it may not mean tests are passing. Test automation scripts need to be validated now and then as well. It also highlights that a test automation script that fails now and then could be more trustable than a test that never fails at all.

While it is a good practice to catch the exceptions and exit gracefully after the automated test execution, it is not necessary to boil the ocean to capture every single exception and not focus on the main job in hand – to validate the functionality of   the test script using test automation. A good test automation engineer does not hold on to poor automation code but is very open to refactoring of the code and is never afraid to throw away poorly written automation  code.

In addition, an automation tool is an enabler. Just having a tool does not make the automation program a success. Need to have a well-defined test automation strategy, framework, and execution plan to make a program  success.

Like AI is augmenting human intelligence, automation scripts can never replace good manual testers, they augment the manual testing. Automation is not a record-and- playback or a linear and simple hard coded program. They are pragmatic software code and they need to be tested just like an application. You need to test the test automation code. It could be done manually or one can use an automated program to test the automated script as well (to reduce complexity and  time).

The proof of the pudding is in the level of maintenance consumed by the automation programs. The lesser the maintenance, the better the outcome. The second value adds the number of defects found in regression because of the automation scripts. The third value adds the synthetic test data created because of running the test automation scripts. This way an automation script can be used by the testing team  as a data generation tool as  well.

Now, let us see some of the good tips and tricks to consider whiledoing test automation as a practitioner.

Some tips and tricks to follow while doing test automation

Some tips for strategizing right include having a sold test automation process as follows:

• Analysis & Design ? Scripting & Config ? Definition of Parameters ? Mgmt.  of Params ? Scenario Collection ? Validations ? Testing of Scripts ? Script Execution ? Review of Results ? Result  Communication
• It is also important to take care of the following earlier by the project/program management team.
• Get executive buy-ins and support - Have a champion sponsor for automation to help you through the valleys, peaks and downhills.
• Automation is a disciplined profession. Have objectives set clearly. This is very important.
• Have good collaboration and cooperation with other  stakeholders.
• Learn from mistakes of self and others. Always learn from others and external learning – Both from failures and leading  practices.
• Have  a  good  team  building  between  automation  engineers,    testers, developers, business analysts, and the project teams.
• Celebrate simple wins and successes. Have a good tooling strategy: - Have an internal and external advocate/evangelist for the test automation tool/framework. - Even test automation tools have bugs and restrictions so beware of the same and factor in during your test automation  strategy. - The automation test suite needs to be flexible and capable of letting users select automation pack based on options and  configurations. - This makes the test suite efficient and permits the scripts to run faster. - It is important to spend the time of training on the tools and learn about the usability and features of an automation tool. - Exploratory test automation is a good choice to consider.

.......

.......

.......

.......

THIS IS INTENTIONALLY LEFT BLANK

.......

.......

.......

.......

Automation tools and methods

In the first two chapters, we covered various generations of test automation, various methods of test automation to follow and a reference to some of the tools to consider, in addition to Selenium. As a recap, let us look at what are the tools and methods we can look at.

Some of the ways we can automate are as  follows:

Linear Test Automation:

• a.   Use of record and playback-based code or a simple procedural  code

Structured Test Automation:

• a.  A simple structured control block-based test automation with the use  of constructs such as ‘IF-THEN-ELSE’, ‘WHILE’, ‘FOR’, ‘SWITCH’, condition-based control structures. Something like a VBA-based automation.

Data-driven Test Automation:

• a.   Use of data in a spreadsheet, control files or a database. Testing logic  is driven by the data given such as a login id, location, data type,  etc.

Keyword Driven Test Automation:

• a.   Use of various keywords to derive the test automation  logic.

Hybrid Test Automation:

• a.   Use of one or more of the automation type such as a combination        of Data and Keyword Driven or Data-Driven automation using an automation tool giving only Structured programming constructs (such as an automation IDE).

Behavior Driven Development(BDD-based Automation):

• a. Use of simple user behavior actions using a language such as Gherkin and tools such as Cucumber or RSpec to generate automation  code.

Model-Based Test Automation:

• a.   Use of Domain/Functional Model to perform Test  Automation.

Intelligent Test Automation:

• a. Automation framework that performs DO-CHECK-LEARN-ADAPT- THINK-CHANGE type of test automation  maintenance.

AI-Driven Automation:

• a.   Taking intelligent test automation to the next level  using AI/ML/DL/NLP/NLG, etc.

Visual Automation:

a.      Automation-based on the UI content  present

As far as tools available, we have covered all of them in detail in earlier chapters. Now, let us understand another topic that is picking up a lot of interest in the tester’s community in the next section – Visual Regression.

Use of Visual Regression in test automation

Visual regression is a technique focusing at the UI level to check whetherthe UI components and behaviors are predictable. It checks for the screenshots for predictable behavior; and in case of deviations from expected behavior, the tool can give hints on what is changing. The regression tool is very good for checking the style element changes in a CSS in a web application. This is very useful for testing stable UI apps with predictable behavior and it works well for testing of web applications. Sometimes, Visual CSS regression testing or comparison of visual differences of a website is also considered a Visual regression. In some cases, Visual regression can be used to do A/B testing to choose the right visual elements to be rolled out to the users.

Visual regression testing can also check for comparison of various web pages side  by side for various web browsers on different platforms, OSes and versions with different screen configurations as well. This gives a way for the testers to validate  the actual behavior of the application against the expected behavior. The aim is to check for visual flawlessness in terms of HTML, CSS, and JavaScript changes in your application in terms of visual aesthetics, user experience, and functional   behavior.

Visual regression testing can be a good choice to check for cross-browser testing and there are plenty of tools available for doing automated visual regression testing. Some of the tools include:

●       Applitools: This integrates with other tools such as SmartBear  CrossBrowser

Testing and allows you to map with Selenium, Appium and other frameworks.

●       Selenium: With the headless browser testing options and the ability to take screenshots, Selenium can be a very powerful visual regression tool. Many programming skills is needed though. Some of the successful tools leverage Selenium as a base for their visual regression testing capabilities. Some of the Selenium-based tools include:

Selenium Visual Diff / Gemini/WebDriver CSS: Use JavaScript on Selenium

Visual Ception: Use PHP on Selenium

Huxley / Needle: Use Python on Selenium

Wraith-Selenium: Use Ruby on Selenium

●       Phantom CSS: This excellent tool uses various JavaScript frameworks  to perform visual regression testing. A strong JavaScript programming capability is needed for using  PhantomCSS

Wraith: Uses Ruby with PhantomCSS

dpdxt: Uses PHP with PhantomCSS

Grunt & Snap and Compare: Uses JavaScript with  PhantomCSS

Visual Regression is getting very popular and is getting help with advanced tools that permit script-less automation, leveraging Artificial Intelligence (AI). Now, let us understand what we mean by script-less test automation in the next   section

Script-less test automation

Script-less test automation refers to automating tests without the need to do programming. The example would be to automate say a web application without      a need to write a Selenium of a QTP code to do the end-to-end automation. This could be done through a tool using workflow or using behavior-driven test design tools (BDD) such as Cucumber with the help of a script-less automation tool such as Tricentis Tosca or Worksoft Certify to generate test automation. This is very useful for the team as the business domain experts can help automate the test suite as early as the application requirements evolve. The components can be created quickly and they are in a position to be reused as the scope grows progressively. The ability        to modularize the test cases also helps in easy maintenance of the automated test scripts.

Let us see some benefits of the script-less test automation  scripts.

• These scripts-less test automation suites record the user interactions in a  web or mobile application and generate the scripts automatically. The tools can also read the requirements and test case documents and generate test automation suites using Natural Language Processing and AI and Machine Learning.
• The script-less test automation suite can reduce the overall effort of automation. However, it is advisable for a good test automation engineer    or a domain/business specialist to validate the output produced to ensure   it is fit for purpose, to refactor and make the code relevant for the scope in hand. This is because a script-less automation framework is not 100% code free. There will be a need to customize it. The automation testers who are    in charge of the script-less test automation framework need to be good in programming to ensure a good product is built. A script-less framework is not a simple record and playback as  well.

Key Visual Regression, Cross Browser and Script less automation tools and services

.......

.......

.......

.......

THIS IS INTENTIONALLY LEFT BLANK

.......

.......

.......

.......

Use of Artificial Intelligence (AI) in test automation

AI has come a long way since its inception. Let us see some of the key benefits of using AI in testing. AI has been a hyped term in the industry for some time now. When it comes to automation, AI has been hyped and quoted for various reasons such as

• Automated Defect Detection, Automated Exploratory Testing, Test  Coverage Heat map
• Self-Healing   Test  Automation,   Predictive   Modeling,   Self-Adjusting Regression, Defect and Execution Pattern  Recognition
• Risk and Coverage Optimization, Test Diagnostics for Prescriptive and Predictive Analysis of Test Planning
• Deep Learning for efficiency, Automated Root-Cause Analysis for failures, User Behavior

And

• Sentiment Analysis

Now, let us see how an AI tool gets its script designed. Most of the tools capture the user actions either by capturing it through an agent running in the user  desktop or through a spider/crawler mode to emulate user actions. 

Once the scripts are designed and captured, the organizations can use the AI/ML techniques to build the models and algorithms for the use of AI in testing. The main aim of the enterprises to use of AI in the automation of testing is to optimize resources, effort, costs associated with testing while increasing the quality and coverage of the product being tested with an optimal number of test  cases.

Out of the techniques mentioned above, the following are some of the top techniques used for test automation using AI:

• Monte Carlo Simulation Tree
• Support Vector Machines
• Search Tree/Deep Neural Networks
• Bayesian Classification
• Probabilistic Models

We have understood some of the available techniques for AI/ML. Using various techniques and options available, let us see how an AI-based system can be built using test automation. The following image is one of the examples of how an automation application can be used for test automation using  AI.

There are multiple ways to implement AI for test automation, based on the needs, skills and tools applicable to a particular business problem and domain. Let us take an example of how AI was used for test automation is a very popular  game.

Example

One of the examples of AI in testing can be looked at in the Gaming Industry.    The popular game Candy Crush Saga was tested with the use of an AI engine for continuous Feedback Loop mechanism. The testing team used AI-driven BOTS to perform testing. All the outputs were fed through in a continuous feedback loop into a Deep Artificial Neural Network built for testing of the billions of permutations across various levels. The testing team used Monte Carlo Tree Search (MCTS) and bots built by a Hybrid Test team (150-200 + Testers) with unique skills across the domain, technology and testing ran test automation scripts. The testing team used Data Scientists for Domain Knowledge, Fun (using historic info and User Behavior, Game Balancing). Test automation scripts looked for Regular Crash Testing through automation, and leveraged automated scripts to do performance testing and regression testing. The team also leveraged AI upgrades for the AI Bot for regular testing.

The key take away from this is to let AI do exhaustive testing through automation and advantage humans to perform qualitative and cognitive aspects. The automated scripts need to Be Creative and Business/Customer-Centric and the automation should be done at the earliest for automation with a long-term  focus.

Another key focus area in the industry is to use test automation for rapid quality delivery using Agile and DevOps projects. Let us see how we can leverage test automation for DevOps Engagements.

Test Automation for DevOps Projects

There is a standard question from testers about AI Testing, DevOps and other new Software Engineering evolution. “Will Automation / AI / DevOps remove the need for Manual Testers?” A simple answer is “No”.Automation, AI, or DevOps is not going  to replace Manual Testing as a discipline. Firms implementing test automation or AI- based testing or continuous testing through DevOps have a vision for better product quality workable by brilliant  testing.

Test automation for DevOps or use of AI for test automation is an enabler for the following:

• Reduction in IT Budgets
• Efficient end-to-end delivery
• Lean delivery framework
• Working end-to-end automation framework

Successful businesses today are driven to deliver features quickly using various techniques. With a design thinking and customer-centric, focus with a strong sense  of urgency and a need to succeed in the market quickly. Availability of brilliant tools and innovation aided by software engineering techniques make it possible. All the successful firms of today are driven by technology, as it has become a backbone, enabler and catalyst all in one. Many tech giants are capable of running millions of test cases in an automated fashion in a span of minutes not seconds, using continuous testing and delivery. For example, the Microsoft team does 82,000 deployments per day, with over 28,000 work items created per day and over 2.4 million code commits per month. How can one test so many changes quickly while making sure the product is of highest quality as billions of people use them across the globe? To do this, we need to have effective habits of  DevOps.

Test automation within zero-touch-end-to-end delivery is essentials. Most of the projects are moving towards AI/ML driven delivery. We have tools that allow a developer to auto complete the entire chunks of code. Even for testing, we can       use Cucumber through Gherkin; generate a fully automatable Selenium code that reduces automation code development effort for the automation engineer. DevOps Governance and delivery model is also important, as we need to avoid manual- errors and stove-piped communication due to distributed teams and vendors that can be removed with a good DevOps  practice.

In addition, it is important to adopt a ‘Test-First’ delivery method driven by ATDD or BDD or any such test-driven method to fail fast and fail often to fail early and fix the issues earlier, as shown in the inverted test pyramid. DevOps permits us and there are many benefits in moving towards such a model:

• Rapid , progressive, and useful product feature  development
• Cost reduction in end-to-end delivery through intelligent automation Reduction of manual errors and de-duplication
• Streamlined governance and rapid go-to-market Early and rapid defect identification
• Agile and cohesive team focused on quality  deliverables

Implementing a successful test automation project for AI/ML or DevOps projects is very tricky in organizations having a legacy and monolithic systems and processes. An appetite for a strong culture change is critical for successful DevOps and continuous delivery. Top leadership team needs to sponsor this.

From a testing standpoint, test automation and DevOps can address some of the following challenges, with the current change in landscape of the tech infrastructure:

●       When it comes to Quality Engineering/Test Environments, the DevOps life cycle makes it very simple with real-time, on-demand provisioning done in  a span of minutes and not months and  years.

●       With the DevOps Model, we can reduce the overall test effort in terms of cost and life cycle with automation completion of tasks from design to deployment and support.

NIST – A standards body reported in 2002 that the Cost of Poor Quality (in the US alone) was $59.5 Billion per year with this value increasing annually. COPQ is the biggest waste of money in large established organizations running legacy systems. By some evaluations, the average cost of a production bug is at least $14,000 to $15,000. In some cases, the cost of a production bug can outweigh the benefits delivered by the feature change that resulted in the bug fix or a feature fix that caused it. Given the new customer base of fivevaried generations, having a poor COPQ will result in poorer customer experience and the organization agility will be impacted. DevOps can be a super catalyst  here.

The skeptics questioning the effectiveness of test automation may have some data to support their argument but failed test automation projects are due to poor strategy, design, planning and execution. The tech giants delivering brilliant services have reached their position purely based on their capabilities and skills, leveraging the power of DevOps.

Some key factors to implement for successful test automation practice for DevOps or an AI-based project is as follows:

●       Cohesive, one team mentality between  teams

●       Implementation and use of tools that take away the manual  effort

●       Continuous delivery phases:

1. Superior, automated configuration management  approach
2. Continuous improvements across phases Continuous automation
3. Continuous integration Continuous testing
4. Continuous monitoring

Having a solid governance model and strategy for tools and processes to implement DevOps and leveraging the power of Test Automation will help the organizations achieve its objectives.

Putting together all the learning, what are some of the recommendations a Selenium automation tester should follow? Let us get some tips and leading practices for implementation.

.......

.......

.......

.......

THIS IS INTENTIONALLY LEFT BLANK

.......

.......

.......

.......

Additional Tips to Purse

• Always automate most frequent and repeated test cases Automate scripts that are easy to automate
• Automate scripts that are stable and predictable in behavior  Automate low value-adding and tedious test cases for a manual  tester
• Automate scripts that takes a lot of manual effort but easy to automate using scripts
• Automate scripts that are run in multiple platforms and browsers Automate frequently used business  functionalities
• UI automation focus alone is not very good
• See if script-less, model-based or BDD-based test automation is possible Use test patterns using design patterns and architectural  principles
• Use a test automation framework for your  testing
• Always make the framework portable across platforms and if possible, tools Have a good nomenclature for naming
• Use assertions and screenshots for validations
• Have a very good explanation for the programs in the form of comments Have a good reporting mechanism
• Always keep the scripts to run independently
• Focus on data-driven, keyword-focused or hybrid automation. Do not keep the tests run linearly
• Have a very good quality test data for automation Test more and test often
• Always factor efforts by other stakeholders into your automation estimate such as business analysts, developers, end-users, tool specialists, compliance specialists, etc. The teams doing automation sometimes overlook  this.
• The tool is important but it is not a one-size-fit-all  solution.
• Good automation script needs to have a very good manual test case as a reference. Garbage in – Garbage out is applicable  here.
• If GUI automation can be avoided with API or micro services automation, go ahead with it and avoid boiling the ocean with extensive GUI   automation.
• Beware of Assert vs. Verify usage – Assert breaks the test whereas verify continues. You may want to consider this.
• Build your library of functions and methods as it could be very handy for your application

It is advisable to understand the challenges and document the learning one may have while implementing the project and share it with the wider community.

Conclusion

We covered all the essentials of automation using Selenium until Chapter 10. In this chapter, we covered a good number of tips and tricks to follow while doing automation, pitfalls to avoid, various tools that we can consider, recommendations on strategy and how test automation can benefit through script-less automation, visual regression automation, use of artificial intelligence, and contribute to DevOps mode of delivery. As mentioned in the chapter, test automation is a discipline and it takes a lot of good practice to become a solid expert. If you learn from the mistakes made by self and others, implement the leading practices, follow a disciplined strategy and approach, use the tips and tricks for your test automation, you are all set for a very successful career. In the next chapter, we willcover one more important aspect of getting a dream job you deserve. How to ace interviews? What are some tips and preparation one needs to do to perform well? Well begun is half done they say, to do well in the interview, you need to prepare well. We will coverthis and other aspects in the next chapter.

Questions

• What are some top tips to consider while using Selenium for Test Automation? What are some tips to consider while choosing test  automation?
• What are the top myths attributed to test automation? What does Visual Regression mean? How is it useful? What is the script less Automation? What are the  benefits?
• What are some of the benefits of manual testing over test automation? Where
• is manual testing more reliable than automation? How will you use AI in test automation?
• How can test automation help DevOps delivery model? What do you mean by purposeful test automation?
• What do you mean by Intelligent Test  automation?
• What is a self-healing test automation code? Can you give some  examples?
• Is 100% test automation possible? What is your take on the  same?

Hope you enjoyed the excerpts from Chapter-11. What are your comments?

Book Availability

The book is available at the following sites

• Australia ==> Amazon ==> https://www.amazon.com.au/dp/B082KFY17B
• Austria ==> Amazon ==> https://www.amazon.de/dp/B082KFY17B
• Brasil ==> Amazon ==> https://www.amazon.com.br/dp/B082KFY17B
• Canada ==> Amazon ==> https://www.amazon.ca/dp/B082KFY17B
• Czech Republic ==> Amazon ==> https://www.amazon.de/dp/B082KFY17B
• France ==> Amazon ==> https://www.amazon.fr/dp/B082KFY17B
• Germany ==> Amazon ==> https://www.amazon.de/dp/B082KFY17B
• India ==> Amazon ==> https://www.amazon.in/dp/B082KFY17B
• India ==> Amazon ==> https://www.amazon.in/dp/B082KFY17B/ref=dp-kindle-redirect?_encoding=UTF8&btkr=1
• India ==> Amazon ==> https://www.amazon.in/Science-Selenium-Automation-Framework-English-ebook/dp/B082KFY17B
• Italy ==> Amazon ==> https://www.amazon.fr/dp/B082KFY17B
• Japan ==> Amazon ==> https://www.amazon.co.jp/dp/B082KFY17B
• Japan ==> Amazon ==> https://www.amazon.co.jp/Science-Selenium-Automation-Framework-English-ebook/dp/B082KFY17B
• Mexico ==> Amazon ==> https://www.amazon.com.mx/dp/B082KFY17B
• Spain ==> Amazon ==> https://www.amazon.es/dp/B082KFY17B
• Switzerland ==> Amazon ==> https://www.amazon.de/dp/B082KFY17B
• UK ==> Amazon ==> https://www.amazon.co.uk/dp/B082KFY17B
• USA ==> Amazon ==> https://www.amazon.com/dp/B082KFY17B
• USA ==> Amazon ==> https://www.amazon.com/Science-Selenium-Automation-Framework-English-ebook/dp/B082KFY17B
• USA ==> Barnes & Noble ==> https://www.barnesandnoble.com/w/science-of-selenium-kalilur-rahman/1135502525?ean=2940163963881
• Brasil ==> Livrariacultura ==> https://www3.livrariacultura.com.br/science-of-selenium-2012821784/p
• Canada ==> Indigo ==> https://www.chapters.indigo.ca/en-ca/books/science-of-selenium/9789389423242-item.html
• Canada ==> Indigo ==> https://www.chapters.indigo.ca/fr-ca/livres/science-of-selenium/9789389423242-article.html
• Germany ==> WetBild ==> https://www.weltbild.de/artikel/ebook/science-of-selenium_26964037-1
• Germany ==> WetBild ==> https://www.weltbild.de/artikel/ebook/science-of-selenium_26964037-1
• Global ==> Google Play ==> https://play.google.com/store/books/details/Kalilur_Rahman_Science_of_Selenium?id=-ofCDwAAQBAJ
• Global ==> Review Site ==> https://www.goodreads.com/book/show/49289577-science-of-selenium
• Global ==> Scribd ==> https://www.scribd.com/book/439434766/Science-of-Selenium
• Global ==> Textbookx ==> https://www.textbookx.com/book/Science-of-Selenium-Master-Skills-Needed-to-Become-a-Top-Test-Automation-Guru-with-Easytofollow-Selenium-Examples/9789389423242/
• Holland ==> Bol ==> https://www.bol.com/nl/p/science-of-selenium/9200000126647725/
• India ==> BPB Publications ==> https://bpbonline.com/products/science-of-selenium-master-web-ui-automation-and-create-your-own-test-automation-framework
• India ==> PayTM Mall ==> https://paytmmall.com/science-of-selenium-RKC0003512361_882733-pdp
• India ==> SapnaOnline ==> https://www.sapnaonline.com/books/science-selenium-master-web-ui-kalilur-rahman-9389423244-9789389423242?position=2&searchString=science%20of%20selenium
• India ==> Snapdeal ==> https://www.snapdeal.com/product/science-of-selenium/626675025728
• Japan ==> Rakuten ==> https://books.rakuten.co.jp/rk/7a977206400b37fb90e0057fc90225a7/
• Portugal ==> FNAC ==> https://www.fnac.pt/livre-numerique/a7376163/Science-of-Selenium
• Saudi Arabia ==> Google Play ==> https://play.google.com/store/books/details/Science_of_Selenium_Master_Web_UI_Automation_and_C?id=-ofCDwAAQBAJ&hl=ar
• Saudi Arabia ==> VitalSource ==> https://www.vitalsource.com/sa/en-us/products/science-of-selenium-master-skills-needed-to-kalilur-rahman-v9789389423242
• Serbia ==> Kombib ==> https://knjige.kombib.rs/science-of-selenium-master-web-ui-automation-and-create-your-own-test-automation-framework
• Spain ==> FNAC ==> https://www.fnac.es/livre-numerique/a7147532/Science-of-Selenium
• Sweden ==> BOKUS ==> https://www.bokus.com/bok/9789389423242/science-of-selenium/
• USA ==> Tattered Cover ==> https://www.tatteredcover.com/book/9789389423242
• USA ==> Walmart ==> https://www.walmart.com/ip/Science-of-Selenium-eBook/466432367

Note:-

Title Images are created using CANVA tools. Authors of the quote referred where known. Most of the information shared is generic and available in various forms in the Internet. Respective trademarks are owned by corresponding firms. Opinions about tools highlighted are from a personal experience standpoint and in no way reflect the views of my current or past employers or clients.

#WhatInspiresMe #Automation #Selenium #KRPoints #TestAutomation #DevOps #AITesting #NewAgeTesting #MobileTestAutomation