3 reasons to go Agile and to Automate your tests!

Let’s take a quick look at why companies are adopting agile development, challenges to face when testing mobile apps and how automation can help achieve a continuous delivery method.

Why do companies use agile development?

Using agile development means to release product updates more frequently, adapting to the changing priorities on the go, while increasing productivity and the quality of the product. All of this will also impact directly how and when testers test the software. Here’s a graphic from “10th Annual State of Agile Report”.

What are the benefits for companies that implement agile?

There are three main benefits that have dominated for the past 5 years in the same research:

1. It’s easy to manage changing priorities,
2. Increased team productivity
3. And project visibility

If you use waterfall development model, you will have to wait for developers to finish writing the code in its entirety before the code can move on to testers.

With more agile methods like continuous integration or continuous delivery, dev teams and QA teams will work at the same time. As soon as developers write a piece of code, fix a feature or add a feature, the testers can run the tests to check if everything is still working. 

Typical agile development cycle

Along the lifecycle of a software or app, testing is done often and after determined code writing. This way you can build a functioning app from the very beginning.

At different stages of app development, you will have to check for different things, running different kinds of (automated) tests.

1. The first set of tests are unit tests and regression tests which are run on a few devices before the build.
2. The second set of tests are run after the build. These tests are typically integration tests, stress tests and performance tests and need to be executed on a larger number of devices because you’re about to release your app.
3. The last set of tests are run after the release. These tests are a continuous set of tests and are meant for monitoring, back-end connections and if needed, you can also integrate a crash analytic tool like HockeyApp.

Testing isn’t just QA anymore, but it becomes a part of development and operations, also known as DevOps. We will have a DevOps-QA team that is responsible for the software, in every stage of its lifecycle.

Everything we have said until now of the development cycle is not really restricted to mobile devices. Desktop applications and software have the same development cycle. Mobile development has its own added obstacles that need to be overcome.

Challenges of mobile app testing

Compared to classic software testing, mobile application testing has some specific challenges. Manual testing is no longer a sustainable option when it comes to mobile devices, especially with Android, since more than 24.000 different devices have been detected until August 2015.

It's quite impossible to perform manual tests on all Android devices. Users would face long waiting times between releases – executing manual tests means to test one device at a time, performing the same actions over and over again. It’s not only time consuming, but wearisome as well, leading to mistakes. Apart from the device fragmentation you can find other types of fragmentation:

• Many different OSs
• Many OSs versions
• Device models fragmentation
• Wide range of screen sizes

One other challenge is the hardware itself. Emulators and simulators cannot substitute physical devices. You won’t get real performance stats, because you don’t really see what’s happening to the device like: battery drainage, overheating and the app crashing due to previously mentioned reasons. You have a few options here: you can buy the devices yourself or you can subscribe to a cloud service where you can find hundreds of devices, providing both older and newer models.

App quality has a direct impact on the app’s success. And who decides how good your app really is? The users are the ones who will judge your app and they are unforgiving: they get annoyed by even the smallest bug and they will voice out their opinions, especially the negative ones. 58% of users will read the reviews of an app before installing it, if there are many negative reviews, the app just won’t be installed. This will result in decreased revenue. So why give users a reason to write bad reviews?

To make sure to release a-nearly-flawless app, you’ll need to test more. Automated testing allows you to perform more tests while reducing your testing time.

How can you benefit from automation and what are the challenges?

Mobile test automation plays a key role in continuous integration and especially in continuous delivery. The more a development cycle becomes agile, the more automated tests have to be executed.

In this environment, testing should be one of the most important tasks in the development cycle and it needs to be done often, instead of just the last mile before the release, with no time for further iteration.

When we speak about automation at this point, we talk about functional UI test automation, so basically we check what the user sees and touches.

One of the main benefits of test automation is the reach a wider range of devices with less testing time, enabling shorter release cycles. One other benefit is that you can test as deep as the network connection and many edge cases.

The initial investment to implement mobile test automation in the development cycle and the acquisition of know-how is something that you’ll have to take into consideration, better sooner than later. Your effort will show you a high ROI soon enough.

With automation you can also scale your testing which is something you cannot do with manual testing. The main downside is that you can only use test automation for about 80% of your testing; there are certain things that you have to test manually. Integrated testing, which is a combination of automated and manual testing, will still save up to 70% of testing time compared to performing only manual testing.

How do I implement mobile test automation?

To start with test automation you’ll need test scripts that cover your test cases. The scripts can be written in different programming languages using different frameworks. You can find more information about the frameworks TestObject supports here. To speed up your testing we suggest using Appium, which is the mobile specific framework derived from Selenium.

It actually is the most versatile mobile UI testing framework supporting many client libraries (Python, Java, JavaScript, Ruby on Rails, PHP, C#). The strong suit of Appium is that it is cross-platform and cross-technology: think about having to write just one test script and running it on iOS, Android and Firefox OS and it will also work with both native and hybrid apps. If you have to start somewhere, then start with Appium!