Developer’s Perspectives in Software Test Automation

It is difficult to overestimate the developer’s prospects in this area. Like bioinformatics, the development of autotests is at the intersection of two disciplines, and the skills of a programmer are increasingly coming to the fore.

Historically, specialists with experience in manual software testing come to automation. At some point, they realize that routine operations can and should be automated, or they just want to try themselves in a new quality. The experience that manual testers bring to an automation project can be overwhelming. No one knows better than them the capabilities and weaknesses of the product, the most time-consuming test scenarios, the environment in which the product operates, as well as the wishes of users and plans for the next releases. As a rule, a good manual tester clearly understands what exactly he wants to automate, but sometimes difficulties arise with writing autotests. Why? Because autotests are the same software product as the product they are designed to test. You need to think over the architecture of the autotest system, the mechanisms for launching them (Continuous Integration, CI), and, most importantly, you need to write good code. In fact, this is often not easy for a manual tester. After all, here you need to create a full-fledged project that can be integrated into CI, modified, extended, and reused. To do this, you need programming ability, accumulated experience, and full of bumps.

A manual tester with a deep understanding of the product and testing methodology and a developer with programming experience complement each other perfectly. The first one is strong in setting the problem (use case -> test case), the second in its implementation. The question arises: why are there many manual testers among Automators? There are several reasons for this:

• The path from manual testers to automated testers was clearer, and the coding requirements could be lighter.
• For developers, the transition to automation is in a sense a break in the pattern: “So I was developing a product, and now I will be a tester who writes autotests — what kind of developer am I now?” In other words, there is a fear of losing qualifications, turning into a manual tester, as well as an unwillingness to dive deeply into the theory and practice of testing.
• Expectations of less interesting tasks compared to the development of the product itself and doubts about professional development.
• Fear of losing in career growth and salary.

Let us examine these points.

Autotests are the same development product as the product for which these autotests are created. If a developer goes to the position of an Automator in a dedicated autotest development group, he remains on the path of writing program code. Yes, he needs to have an idea of software testing, but basic knowledge can be gained relatively quickly by flipping through any software testing manual (for example, this book). Of course, an Automator needs to understand the product for which he writes autotests. But mastering the product at an acceptable level is often easier than learning how to write good code. Knowledge of the product and testing methodology will expand as you become familiar with bugs written to the product and communicate with manual testers. A developer won’t stop being a developer, won’t turn into a manual tester. This is not his way. The developer’s way is to write good autotests.

The tasks of the automation engineer are interesting and often difficult. First of all, it is worth remembering the famous Fowler pyramid. Unit, integration, end-to-end tests imply a thoughtful approach to the structure of tests and the choice of tools in accordance with the functionality of the products for which we are writing autotests. If we talk about products developed in Veeam, then the Automator will need to work with REST, WebDriver, Microsoft SQL Server, Amazon Web Services, Microsoft Azure, VMware vCenter, Hyper-V — the list is not exhausted. Each of the clouds and hypervisors has its own API and skeletons in the closet. Sometimes you have to write code in various programming languages, use stubs, semaphores, create your own wrappers, etc.

One and the same problem can be solved in different ways, and the Automator looks for the most effective solution. Here is just one example — a scenario implemented for the Veeam ONE product. One of the components of the product is Business View, which allows you to group virtual infrastructure elements according to various criteria. There are a lot of criteria and options for combining them, so manually checking this functionality takes a lot of time. Writing head-on autotests simulating the actions of manual testers would be ineffective: tests for the graphical interface of desktop applications are usually complex and time-consuming to develop, fragile, difficult to modify and take a long time to complete. We found another solution: since user actions in the UI are interpolated into SQL queries against the database, we use SQL queries to create categories and groups. This allowed us to cover all properties and operators involved in Business View with autotests within a reasonable time frame.

How can we make sure that tests that are run in parallel to reduce the overall run time do not interfere with each other? How to measure product coverage with Autotests? Code coverage analysis? Analyzing the coverage of custom scripts How to track service drops? How to integrate autotests with reporting frameworks? How to integrate autotests with build builds in CI? How do I run tests on pull requests? And many many others.

You need to pay attention to the quality of the autotests themselves. Who watches over the watchmen? Which autotest can you trust? The autotest should be effective in terms of the “amount of effort expended on it / the result obtained”, autonomous, stable (non-fragile), fast, reliable (no false positive and false negative). The autotest should contain clear, good code from the point of view of the programming language capabilities, so that this code can be easily extended and changed.

The beginner Automator proceeds to write simple tests according to agreed plans/approaches/tools. Solves the simplest problems, learns to cope with difficulties, finds solutions, acquires knowledge and skills. The tasks are becoming more and more serious, the credibility of the automated operator is growing. If the Automator does not stop growing, then after a while he grows to the development of complex technical solutions and deeper participation in the discussion of strategies and priorities in the development of autotests. One day, he takes under his wing automation beginners who want to add value and grow.