Digital Image Processing for Test Automation

Introduction

Image processing is a way to convert an image to a digital aspect and perform certain functions on it, to extract useful information from it. Image processing basically includes multiple scopes/steps:

Image Acquisition

This is the first digital step in image processing. In the field of test automation, the image acquisition is fairly easy through taking screenshots of webpages, mobile apps during test exaction?

Image Enhancement

Procedure of improving the quality and information content of original image before processing. Common practices include contrast enhancement, spatial filtering, density slicing, etc..

Image Restoration

Image Restoration is a function of taking noisy image and measuring an unused, new image. Exploitation can occur in many ways such as action blurring, sound and camera focus, the purpose of image restoration techniques is to reduce noise and reclaim the loss of decision.

Coloring Image Processing

The understanding of the physics of light, color vision phycology for the classification of objects in an image. Color for the purpose of separation image processing process is used.

Wavelets Processing

Wavelet transform is an effective technique for image representation. The wavelet transform allows for the investigation of multiple brushes/forms of the image.

Image compression

Image compression is a type of data useful pressure digital photography, reducing their costs last or spread. Processes can reap visual benefits awareness and asset data image assets to complex effects related to normal pressure strategies.

Optical Character recognition (OCR)

the electronic or mechanical conversion of images of typed, hand-written, or printed text into machine-encoded text, whether from a scanned document, a photo of a document etc...

All these processes and functions can be achieved by a set including and not limited to the following libraries are involved in performing Image processing, Scikit-image, OpenCV, SciPy, Pillow, Matplotlib.

Image Processing for test automation (Image comparison & Visual Regression)

Regression Testing is used to verify that any system changes do not interfere with existing features and/or code structure. Visual Regression Testing applies the same logic but confines testing to the visual aspects of the software. In other words, it checks that code changes do not break any aspect of the software’s visual interface. A visual regression test checks what the user will see after any code changes have been executed by comparing screenshots taken before and after code changes.

How Visual Regression Testing Works?

1. Visual tests generate, analyze, and compare image snapshots to detect if any pixels (visual diffs) have changed.
2. The first step the test code is executed, an initial set of screenshots is recorded (to act as a baseline against which all further changes will be compared).
3. After setting the baseline, the test runner runs the test code in the background. Anytime a change is identified, a screenshot is captured. Each screenshot is compared to the baseline image corresponding to that part of the code and the software. If differences occur between the images, the test is considered failed.
4. Once the test code runs completely, a report is generated automatically. A reviewer then reviews all the images that have been diagnosed as changed from their baseline. Some testing tools generate reports highlighting the differences between baseline and final images, as detected after the test execution.
5. If these image differences are caused by bugs, developers can fix them and rerun the test to check if the fixes worked. Suppose subsequent changes in the UI cause discrepancies. Most automation frameworks support visual regression, it requires (1) test runner, (2) code written through driver of frameworks like selenium or cypress.

Demo image procesing server

Image Processing for complex systems [Game apps, AR/VR automation testing]

The approach

• The basic ultimate challenge is that most of apps like games or VR/AR are not easily tested on both manual and automation levels, there is no well-defined test oracle, no locators for automation, etc…
• The approach is to develop a test automation bot that will work as runner for the tests (mostly using adb as Appium will eventually need locators which is not sufficient in our case).
• Using the already developed rest api for audio and image assertion, a tester can easily consume it for making the tests etc…
• In a game where you have a 100 level its better to run bots in parallel (skipping levels) to enable faster results and better test performance (demo https://youtu.be/-yMl_3HdzeI).