Building a Device Lab

It has become more and more challenging to test apps on different devices, with different operating systems and screen sizes. As a developer, designer and a QA person you would want your app to run flawlessly across all platforms. Developing a device lab seems to be the answer of all the challenges being faced today in Testing.

A device lab is a highly functional way to provide handsets, tablets, and other Internet-capable devices to those interested in testing their work across a variety of screens and hardware. Developers, QA and others can use a device lab to gain a better understanding of what users experience when visiting their app from these devices. The examples in this article come from a device lab being set up at Jazz to support remote offices, QA teams, automated testing and more.

Choosing devices

Naturally the first step is to choose the devices you want in your Lab. While it's good to test on the device in your pocket, remember that your audience is diverse so diversity is important in choosing the test devices.

Your personal device probably won’t give you a good picture of what bugs might crop up, how long your app takes to load, or how usable your app's UI is on a typical screen size. ?Here a device lab comes in handy, as it’ll open your eyes to the range of user interfaces, screen sizes and sluggishness that your users experience every day.

Start by analyzing your user base, what type of devices and OS version are being used. This will help narrow down the list of devices and you can then start to prioritize and procure the most used mobile devices for your lab. Google analytics really helped us finding the range of devices and OS versions on which the app is widely used. We made sure to have a good coverage of operating systems, screen sizes and pixel density for an optimized and broader user experience.

Also remember, this is an iterative process. The device that is most widely used right now might get obsolete overtime and you might have to buy a whole new range of devices to cater the user experience. This happened with us with the introduction of the new Samsung Foldable range devices on which the UI broke altogether.

Power and Infrastructure

The most basic architecture of a mobile device lab usually a closet or table, filled with the mobile devices you want to test on. It's a closet where devices are scattered on the shelves, each connected to a power outlet and a wi-fi station. Make sure the surroundings are open and well managed by a security lock and camera to avoid any theft and to have a stable internet connection.

At Jazz, we opted for a close room where the closet is mounted in. The room has a security lock and a camera installed and monitored 24hrs. Our mobile device lab also has a 24hr air conditioning system to keep the atmosphere cool with a stable company internet.

Power management is where things get tricky. Mobile devices run on batteries. If you intend to store a lot of devices in a rack solution, then you need to take some things into account:

• Batteries tend to overheat
• Batteries degrade

Make sure you put devices on a metal shelve (in our case it is a wooden shelve), since these devices get hot over time because of the battery and charging. The metal shelve will disperse heat better. Leave gaps between the devices, to make sure cool air flows between devices.

Over time, batteries degrade. A good solution for this is to invest in a good USB Hub. In a Mobile Device Lab we built at Jazz, we're using multiple USB Hub's attached to a Timer. The timer automatically switches on/off and charges batteries according to the power profile of the device. This means devices will only charge when necessary, while still being synced to the host computer.

Device Setup and Configurations

Once the devices are plugged in and charging the next step involves the device setup and configurations for easy access.

At Jazz, we configured each Android device with a single Gmail and disabled automatic updates and notifications. This helps in the long term as during automation notification becomes tedious to handle so turning them off altogether is a good solution.

Turning auto updates off is important as some bugs tend to be found only in a specific OS version and it becomes easier to replicate these bugs if the auto updates are turned off. We followed the same process for all the HMS and IOS devices available in the lab.

Another important thing that we did to keep a track of the devices is to make a QR code for every device and paste it at the back of the mobile device. This helps in instantly knowing the name and manufacturer of the mobile device, OS version, IMEI number, Appium and System Port and the SIM card number installed in the mobile device.

User Experience

It is important to make sure that the experience inside the lab is pleasant so anyone would want to sit inside the lab and use the facilities. You need to make sure that the devices are properly at place with cables all intact and not falling out. The devices are charged and ready to go whenever anyone needs them.

At Jazz, we make sure that all mobile devices have a stable internet connection and are charged for use. All mobile devices are linked with a google playstore and Apple ID account so that it is easier to download the app that needs testing. All the mobile devices have active SIM cards for OTP related test cases. The QR codes helps easily identify the device IMEI, SIM card number for automation tests.

Running Automated Tests

This part is pretty easy, there's a very good open-source project available that does most of the heavy lifting, called Appium. Simply install Appium on the host computer and after some minimal configuration steps, you will be able to point your tests to this Appium server. Once you start a test, you will start seeing activity on the mobile device. This is Appium controlling the device.

For multiple devices, we are using Selenium Grid to host different devices and with changes in the code and configurations of Appium we are able to run our tests on multiple mobile devices for better coverage. After the tests are executed, an automated email is generated with the test results.

A mobile lab greatly helps the Tech team analyze and make a better product suitable for all available mobile phone manufacturers.