Manual tests to automated terminal tests. A step by step guide!

So you are in payment terminal?DevOps?and have decided to migrate your manual terminal tests to automated terminal tests. You are in the process of choosing your automated testing solution and it is highly likely that you will be asking yourself the following questions:

• To run tests automatically, what exactly must the solution be able to do?
• How do we integrate our terminal, and all surrounding systems with our new, automated test scripts?
• How do we start a test transaction on the terminal?
• Will it be possible to validate the actions of the terminal?

Let’s find out. In this post I will provide a high-level step-by-step guide that can help you with this process.

Manual Tests as Basis

To achieve automated testing, you will need to find ways on how to automate the individual steps of your tests. Therefore, you need to understand in detail what those test steps are and what they do. For example, do you just want to test the general transaction flows and user interactions? Do you also want to validate low level details (e.g. financial messages content,?EMV?APDUs etc.)? A good starting point to answer these questions is to look at your existing manual tests.

>> Tip 1: Use you manual tests as basis to build your automated tests.

You might have a very large number of manual tests. Hence, it might not be practical to look at all those tests in detail. However, instead of looking at all those tests, you can choose one test (or a few tests) that is representative of most other tests. Then, you can use the chosen tests as basis for the process described below.

>>Tip 2: Choose one (or a small number of) representative test as basis.

But how to chose this representative test? You will most likely have a number of basic transaction tests that serve as basis for many other tests. In our experience, about 50% of tests (or even more) are slight variations of the same scenario, maybe with a few steps more or less. The rest of the tests are special cases or scenarios. Focus on those basic transaction tests and chose the one that covers the most common scenario. For example, if you have standalone and integrated terminals, you will most likely have a basic transaction test for either of those. Chose the one which is more relevant to you. For example, if 80% of your terminals in the field are integrated, chose the integrated test.

Vertically Slice through the Test

You have chosen your representative manual test. Now you can analyze it to determine what your automated test solution has to do. To do this, vertically slice through the test so that you can analyze each test step. After that, you can identify the functionality you require for test automation.

>>Tip 3: Vertically slice through an existing test to analyze all involved steps and tools.

By vertically slicing through the test, we mean the following:

1. Analyze each step:
2. Is this step required if the test would be automated, if so:
3. How can the action described in the step be automated? What functionality is required to do this?
4. For example, is there a simulator that can be used? Can you use a?robot,?card probe?or?something else? Do you have to build some custom tool? etc.
5. How can the checking of the expected result be automated? What functionality is required to do this?
6. For example, is the result logged in a log file that can be parsed automatically? Is there an API somewhere that can be called to validate the step? Is it a camera that takes a picture and uses OCR to parse that picture? etc.
7. Identify and create required test automation functionality:
8. Compile a list of the individual pieces of functionality as identified above
9. For each item on the list, determine if you already have this piece of functionality or if it has to be built or sourced (e.g. robot)
10. For functionality that has to be built, define interfaces and requirements and build it
11. Define how your test orchestration tool can interact with each piece of functionality

Proof of Concept

Once the process above is completed, you can finally bring it all together. Your test tool can, for each test step, call those pieces of functionality that you have defined above. We recommend to use an iterative approach where you focus on a selected few pieces of functionality only. Run a proof of concept for these and extend your setup step-by-step. Start with functionality that is required for automation first. For example, you should first automate the pressing of keys, because this is required to actually step through a transaction. Host message validation and similar things are not required for actual automation and can be addressed later.

>>Tip 4: Chose an iterative approach and focus on the steps required for automation first.

Focus on basics first

When you start with automated testing, we recommend to focus on those basic tests as mentioned above. These are the low hanging fruit. If you have 50% or more such tests, automating those will already give you many automated tests. Hence, you will also have more time up your sleeve to migrate the special test cases.

>>Tip 5: Focus on the low hanging fruits: The tests that can be easily automated

When demoing our solutions to clients, I hear questions like the following very often:

• What about fallback to magstripe, can this be tested too?
• What about communication errors and timeouts, can we test that too?

While the answer to these question is almost always yes, they are probably not really relevant at the beginning of this journey. Think like this: Should we stay away from automated testing because we cannot automate those 1% fallback test cases?

Continuous Improvement is a Journey

We see testing as a part of a cycle of continuous improvement. You start to automatically test your software. This will increase the quality of your software because you can test faster and more accurate. In addition, testers are free to create new tests, which will re-enforce this cycle.

Like your software, your test tooling should evolve over time as well. As mentioned above, start with basics first. You can add more sophisticated pieces of automation over time. This is a journey about becoming a little bit better every day. Step by step.

What are your thoughts? Have you automated your terminal testing yet?