Image Comparison Testing: Are selfies taken, sent and received correctly?

With more than 250 million photos uploaded to facebook per day and around 1 trillion photos taken worldwide in 2015 it’s obvious to see the impact that smartphones have on our behavior towards taking pictures. In May 2015 Business Insider highlighted that over 8,796 photos are shared on Snapchat every second. I estimate it’s over 20,000 now in March 2016.

We are seeing the growing popularity of the camera feature not only in social consumer apps, but also more and more in business oriented apps and industries such as healthcare, banking, and even manufacturing, to mention just a few.

With our constant efforts to improve app testing, testmunk obviously cares a lot about improvements in testing ‘camera’ and ‘taken image’ features. In this article we’d like to walk you through a setup that we have used for a social app that has such testing needs. We very much enjoyed our journey to establish this setup, so we felt it was worth sharing with you. Hopefully you can take advantage of it as well.

Image Comparison Possibilities

Before we dive deep into our setup, let’s look at the definition of “image comparison testing”. Our definition lies in comparing two similar images. In testing terms this could be comparing two screenshots, meaning you take a screenshot from a specific UI view now and check it against the UI view in a newer app build later.

The goal of such testing is to make sure the screenshots look completely identical, meaning that all elements are on the screen and that they have the same size and position as the previous “old” UI. Image comparison can also be used to verify if pictures are taken and received successfully (and in decent quality) on different devices. This article will elaborate on the latter form of testing.

Our device setup

For our setup, we are using two Moto G devices on Android 4.4 along with two 5 inch tall plastic figures (“Joker” and “Hangover”) that we set up in front of both devices. These will be the subjects of our photos.

Joker and Hangover with two test devices

In order to test the app’s back and forth sending we used testmunk’s app to app testing capabilities and extended the testcase with a few image comparison steps, which are described in the following paragraphs.

Testflow

Our goal is that we can make sure that if User 1’s device takes a picture, and sends it to a User 2’s device, that it arrives correctly, and looks as it should when compared to a master image. Then User 2 takes a picture, send it to User 1’s device, and we verify that the correct image is being received, by again comparing it to a master image.

You might wonder if this functionality could be tested with just a single device (ie, sending to yourself). Yes, you can, but only partially. You can take the image, send to a different username, then log out, and log back in as the second user on the same device. However, testing with two devices, as shown in the image above, has several advantages:

READ MORE