STAR WARS: CHAPTER 1 OF MOBILE BANKING APP REVIEW

We're shortly to release the 6th edition of the mobile banking app review. In this article, we share our analysis of the store ratings and how comparison of these may soon be irrelevant. To find out more and get exclusive material visit our website.

Store ratings

When we first started the Mobile Banking App Review back in 2017, we assumed the app ratings in the stores would be very revealing as to which were the ‘best’ and ‘worst’ banking apps. But when Apple launched a redesign of their app store in Q4 2017 the star rating became an almost meaningless indicator of the quality of an app. To put this into context, the average rating for current account apps on iOS in 2017, before the changes, was 3.5 stars varying from the lowest rating of 1 star (M&S) to a high of 5 stars (Revolut), whereas today 87% of banking apps are rated between 4 and 5 stars, with an average of 4.5 stars for current account apps.

So, what did they do to cause this impact? Apple did three things;

1. they provided app developers with the option to reset their reviews and ratings when an updated version is released
2. they allowed developers to prompt users to leave reviews and ratings without leaving the app itself; and
3. users also no longer had to leave a written review describing the reasons for their rating.

The iOS store changes resulted in a 75x increase in reviews added

The rate of inflation

It is suggested that in-app prompts can inflate an apps rating, by allowing developers to request the prompt at a time of their choosing. Developers can achieve “sample bias” by zeroing in on their fans and avoid asking users deemed a risk[1]. They can make the request for a rating when users are most likely to feel satisfaction with their app, for example when they have just left in-app chat having been helped or received money into their account.

It has been reported that this simple native review prompt led to an incredible 32x increase in app ratings, and 90% of apps that implemented it saw their average rating increase as a result[2]. We saw increases much bigger than this in our banking apps, with over 75x increase in reviews added.

Covid-19 spike

In the chart below, we can also see a spike in Q1 2020 (during national lockdown) which is likely to be a result of increased digital adoption. Traditional providers such as Nationwide, RBS and HSBC, saw a significant increase in the number of app reviews received in that period. Bank quarterly results support this trend. Lloyds reported 17.2m digital active users, up 700k, with mobile app users at 12.1m (up 1.4m) showing mobile is growing twice as fast as digital . Average app log-ins also rose to 25 per month per banking app customer, up from 22 in 2018 . Similarly, NatWest reported their active digital users were up 0.3m in Q3 2020 compared to Q3 2019, now at 9.3 million.

Following Apple

Google did not immediately join Apple in implementing this change, which has meant for the past three years there has often been a disparity in ratings across operating systems, despite the apps having parity in functionality and feature sets. However, back in May 2019, Google announced it was making a change to how its Play Store app ratings work, too. But instead of giving developers the choice of when ratings reset, it began to weight app ratings to favour those from more recent releases[3].

Android is seeing a fifth of the number of reviews as Apple, but we would expect it to reach the iOS numbers within the next three quarters

More recently, in August 2020, it mirrored it’s rival and implemented ratings prompts but unlike Apple, users are not able to opt out of review prompts altogether[4]. Currently, Android is seeing a fifth of the number of reviews as Apple, but we would expect it to reach the iOS numbers within the next three quarters.

This is likely to lead to ratings becoming more alike across operating systems and therefore, the Star Wars is likely to come to its final end. Where the number of Android 4-5 star ratings is currently at 62%, we soon expect this to reach 80+% of apps, similar to what we see on iOS.

Who do you trust for rating now?

With store ratings becoming synonymous, how will customers differentiate and compare banking providers? Perhaps they will look to Trustpilot for the inside story or rely on the FCA’s customer service survey results to provide a fairer representation? We compared this data to help assess its credibility.

In stark contrast to the similar app store scores, we can see from the table above some significant disparities on Trustpilot, with the fintech apps more likely to have taken ownership of their profile, respond to negative reviews and promote reviews from their customers. The fintechs have an average score of 3.6 compared to 1.8 for the incumbent banks.

The fintechs have an average score of 3.6 compared to 1.8 for the incumbent banks on Trustpilot

Comparing the Trustpilot scores to the FCA survey, we can establish a very strong (0.8) correlation coefficient. A higher Trustpilot score is likely to align to a more favourable service quality metric. For example, Starling Bank is rated 4.5 and 86% of customers are likely to recommend the provider to their family and friends based on service quality. Similarly, at the other end of the scale, RBS has a score of 1.6 and a lesser 46% of customers would be willing to recommend the bank.

Furthermore, the online and mobile banking services recommendation results also correlate strongly to the Trust Pilot scores (0.7).

There is an almost perfect correlation between the mobile banking and the customer service scores published by the FCA (0.9 coefficient).

Are customers assessing service quality, to a large extent, based on the digital services they receive? It is worth noting there are a few outliers to this trend, including Barclays, Halifax and Lloyds where despite low Trustpilot scores, customers would highly recommend the banks’ apps. As people increasingly turn to digital services, capturing and monitoring customer feedback in-app will become more important.

Looking beyond the stars

In our review, we measure and assess the banks’ apps based predominantly on their features and functionality, as well as user experience, age, design and speed. Although our report is not targeted to consumers it allows a much more detailed comparison where differences are highlighted rather than masked by synonymous star ratings. We suspect alternative ratings like Trustpilot and the FCA’s study will get more prominence in the future and so providers should look beyond an easily manipulated store rating.

We'd love to here what you think of the study - reach out on social media to any of the team with your thoughts or feedback.

[1] Tech Crunch, Google Play is Changing How App Ratings Work, (May, 2019)

[2] https://www.androidpolice.com/2020/08/06/googles-new-in-app-reviews-let-you-rate-apps-without-switching-to-the-play-store/

https://www.androidpolice.com/2020/01/11/play-store-tests-include-new-in-app-review-dialogs-and-a-dedicated-reviews-section/

[3] FT Article, Apple: how app developers manipulate your mood to boost ranking (6 September 2020)

[4] https://www.androidauthority.com/play-store-fake-review-problem-1082191/