Do US Neobanks Spark Joy in their Users? AI Gives us the Answer

It’s a good time to take a hard look at US neobanks following up on my hard look at the UK's challengers some months ago. Is the new banking revolution all that it’s made out to be? And what about the traditional banking incumbents – are they making good on their promise to exploit digital opportunities and catch up? I think that the next wave of banking is upon us, but traditional measures, which happen to be well suited to measuring larger incumbents, nonetheless left me thinking I might be missing something. Similarly, neobank's sky-high valuations certainly don't really tell much of a story. So I decided to investigate another way, using non-traditional, AI-based sentiment analysis.

There are many ways to look at neobanks’ successes. I am going to focus on their apps, and see what their users are saying and thinking. This approach is intentionally limited; I’m curious to discover what neobanks are actually delivering to their customers, and how those customers respond. Accordingly, this is in no way a complete analysis of neobanks’ capabilities; rather it is a survey of what their users have to say about their interactions. 

73% of people indicate that they are likely to leave a review after a negative experience.

It can be argued that app reviews don’t accurately represent the entirety of the user base. I agree and fully admit that the reviews left on the app stores are a) biased toward the negative, and b) not representative of the entire population of users. It is well known that app reviews tend to come from people who are dissatisfied. In fact, some 73% of people indicate that they are likely to leave a review after a negative experience. This means that we are looking at sets of customers for both neobanks and incumbents who are less satisfied than average. But in terms of evaluating user opinion, this group does offer an advantage – they allow us to see what dissatisfied users are unhappy about. 

There are ample reviews of digital banks on sites like “nerdwallet.com” that more thoroughly weigh the advantages and disadvantages of digital banking offerings. These scrutinize the banks’ service, fees, sign-up, and app usability. These reviews are suited to correlating the needs of a given digital banking consumer with platforms that meet those needs to varying degrees.

My goal is different. It’s to glean useful information after the fact, from user comments, to see what their experience is after the fact. Incumbents have spent billions refining their services in an effort to embrace digital transformation. How are they doing compared to neobanks with budgets in the hundreds of thousands? The clients are speaking and it’s up to us to listen, even if we acknowledge that those doing the speaking tend to be the “complainers.” Or are they? 

Banking app reviewers are normal people who simply wish to share their frustration. Ignore them at your own peril.

It’s also worth noting that banking app reviews differ from on-line reviews posted on travel or restaurant sites. Here users sometimes have hidden agendas, such as building a reputation as an expert or slamming a competitor with fake reviews. In contrast, with bank apps, a quick read indicates that by and large posters are normal people who are having a hard time, and wish to share their frustration. Ignore them at your own peril. Most banks don't and with a few exceptions respond to clients directly.

As for bots plastering a given platform with fake reviews, it’s a real thing on app stores, particularly in gaming and major tech sites like Facebook. So far, the bank apps seem not to have succumbed. 

This study was originally inspired by a recent report stating that “brand value” for the UK’s incumbent banks showed a loss of $US 2 billion. This was a macro-scale analysis and much of the loss was attributed to the challenger bank Monzo. While this indicates the banking revolution is well underway in the UK, the analysis is a) derivative and b) not people-oriented. It doesn’t capture the way customers are relating to their new banks.

To get closer to client opinion in the US, I went directly to the bank app reviews on the Google Play and Apple App stores in the US. The following table indicates results for the leading neobanks versus the top five incumbents by size. Note that making a selection of the neobanks in the US market is challenging as there are over 40 different banks in play and a paucity of publicly available data to rate them by size. While it’s easy to pick Chime, Simple and BankMobile as majors, others were picked by their popularity and market significance. In the end, I settled on 12 of what I believe to be the most competitive players.

Neobank app ratings compared with incumbents in the US

Top rating of 5

Data as of October 17 2019, *Greenhouse only has an iOS app. **Marcus uses the “Clarity Money” app.

Surprise! The incumbents are netting significantly higher average app ratings than Neobanks!

The incumbents are, on average, giving the neobanks a good beating when it comes to designing software that people rate highly, and netting significantly higher average ratings. This is quite a surprise, as neobanks trade on their sexy user interfaces and ease of use relative to incumbents. If they are going to convince users to switch, it is imperative for them to convert design into high ratings. It seems clear that on the basis of app score alone, the neobanks aren’t bringing anything to the market that would compel switching from an incumbent. This does not consider the number of functions each app contains, which in most cases favors incumbents who have had more time to develop their apps. So neobanks take note, you’re not “killing it” just yet despite your advertising.

Another thing is made clear in this table. The number of reviews indicates that when it comes to putting apps on phones, the incumbents dominate – none of the neobanks even come close to the volume of the major incumbents. While this may not be a surprise, the biggest neobank is more than 2 orders of magnitude smaller in review count than the average incumbent. This could be interpreted as a rough indicator of how far neobanks need to go in terms of landing clients.

While these numbers are tantalizing, it is impossible to draw firm conclusions. The number of reviews doesn’t necessarily correlate to the number of active accounts. In addition, when tallying the number of reviews, we must remember that some of these apps have had very long lives on the app store and certainly predate the arrival of neobanks. Size isn’t everything, what we’re really looking for isn’t so much quantity so much as quality.

As for rating scores, so far the incumbents appear to be winning, but what do people really think of their apps when they write a review? To find out more, I put AI-driven sentiment analysis to work on the comments available on the Google Play store.

(If you’re not familiar with these programs, in broad terms they measure sentiment--positive or negative--and emotional content, specifically Joy, Anger, Disgust, Sadness and Fear.)

My methodology is limited by the demonstration software I used (https://natural-language-understanding-demo.ng.bluemix.net/), which was not designed to take in large data sets. Given this limitation, I capped the number of reviews for each app to 30 to ensure that each had the same sample size. Of course, analyzing several hundred would be more statistically relevant. Note also that only the Google Play store gives access to a large number of reviews on a desktop. Apple’s app store only allows reading all reviews on their mobile app. Given these factors, the sampling won’t be definitive, but it does open avenues for further inquiry.

Sentiment for App Reviews on Google Play

Positive sentiment signifies an overall sense of satisfaction

Average Sentiment for Neobanks: -0.02

Average Sentiment for Incumbents: -.52

But wait! User sentiment tells a different story. Some neobank apps are slaying the competition.

Sentiment analysis attempts to understand what clients are telling us at a deeper level as they communicate about their experiences with these apps. From the sentiment perspective it’s clear that Aspiration, BankMobile, Marcus, Moven, Simple and SoFi are not just delivering on their promise to improve banking, but slaying their competition. If we’re looking for quality in experience over quantity these neobank apps are delivering. That users had such positive sentiments is revealing and is at the very least indicative that something transformational is happening with the relationship these banks have with their customers.

Buyer beware. Some neobanks are clearly not worth the bother.

But not all of the neobanks are able to deliver. For banks that were supposed to make banking easy, some seem to have rivaled the negative sentiments of incumbents. Ally, Chime, GoBank and T-Mobile all scored fairly negative sentiments from their users that matched or exceeded the negative sentiments associated with the incumbents. This objectively makes one wonder if using some of the neobanks is worth the bother when both sentiment and rating are all against them. Buyer beware.  

Shockingly, not a single incumbent scored a positive sentiment.

What is interesting, if not shocking, about this analysis is that not a single incumbent scored a positive sentiment from their clients. This speaks volumes as to the sentiment of customers toward these large institutions. Bank of America is noteworthy for having the lowest negative sentiment among incumbents which is perhaps indicative of its significant efforts in digital transformation. 

So what’s going on with these numbers? It doesn’t appear to add up that apps with lower overall scores would have higher levels of user sentiment. Reviewing the data it was clear that the motivations for leaving reviews in the app store were different between the users of neobank apps and incumbents.

Neobank users actually posted how much they liked their bank. So not all app reviewers are complainers.

Incumbent users almost universally posted reviews that complained about features of the app that didn’t work, whether it be a balky fingerprint reader or a new version with glitches. Neobank users, on the other hand, were actually motivated to post how much they actually liked the bank. While they may have had issues with functions in the app that made them give it a lower rating, in their written comments they generally had something good to say about their experience with the neobank. The difference between the two groups was dramatic and I do believe says something positive about the neobanks’ relationship with their customers. Their users are reacting favorably to their services and communicating it back through their comments.

So now to the bigger question at hand, do neobanks spark joy in their users? Yes they actually do, but certainly not all of them. Marcus and Moven users scored significantly higher for the emotion Joy than all others and helped boost the average. I found a considerable number of neobank users simply broke with the convention that app reviewers are pre-disposed to be negative and left genuinely warm and glowing reviews. They were happy, so happy that they exceeded whatever threshold of happiness is required to compel them to write a positive review. None of the incumbents garnered such positive written comments.

Analysis for the emotion “Joy” for app reviews on Google Play

Average Joy for Neobanks: .58

Average Joy for Incumbents: .54

Yes! Neobanks really do spark joy in their users!

I am the first to admit that the emotional content scores of limited samples should be taken lightly. I do think, however, that the more robust sentiment analysis says something worth listening to. If neobank users can be compelled to leave such positive sentiments for these new services one has to ask if they are relating to their clients on a different level or is this simply a question of expectations?

One of the arguments that can be made against this analysis is that the reviewers suffer from “self-selection bias” and that the neobank and incumbent groups actually represent two different groups of people. This might be true to some degree as neobank users left comments that made them appear younger than incumbent bank users. Most incumbent bank users seemed to expect their banking app to work better and left few other clues. Selection bias may have had some impact, but neobanks’ users seemed to feel compelled to write about positive experiences with new services they were using which seems to point to a more analytic process than one of age segregation. Moven and Marcus are examples of this as both have budgeting or money management services built into their apps that may have contributed to their client's joy.

Neobanks are onto something. That they solicit such positive sentiments for their apps is impressive.

While the battle has not yet been won by neobanks there are at least clear signs from some of the better neobanks that they’re onto something. That they elicit such positive sentiments for their apps, which ultimately define their overall service, is impressive and noteworthy. It would be easy to just look at these results and call them a fluke. In fact, that’s exactly what many neobanks hope incumbents will do. This will give them the time they need to persuade even more clients that their services are somehow different. Whether their app has the best rating, or has the most functions may not matter if your fees, penalties or other services make users unhappy.

Incumbents take note, people really can like their bank.

Incumbents take note, people really can like their bank. The paradigm that people only complain about their bank is shattered. When compelled people can spread the joy and that is the problem for incumbents. The same people who felt compelled to write positive reviews are going to tell their friends and family that there’s another way to bank. Perhaps that, above all, is the biggest threat to the incumbents. Not a mass migration of clients to neobanks, but a slow but perceptible trickle.

One more critical piece of data available to us on the app stores: the number of versions of apps that the banks are producing. Neobanks are producing new versions of their apps every week or two while incumbents may be on a one month cycle or more. With higher rated apps it’s clear that neobanks aren’t just fixing bugs, but doing what they do best, rolling out new features and better service at a blinding rate. The neobanks have to. They are trying to catch up with the features that incumbents launched years before and at the same time trying to deliver a customer experience that further differentiates their services.

 Bank of America, the leading incumbent, had 6 versions in 154 days or 1 new version every 26 days, while Clarity had 6 Versions in 87 days or 1 every 14 days 

Source: Apple App Store

Incumbents are delivering on their promises of digital transformation, but haven't made clients like them.

So are incumbents delivering on their promises of digital transformation? Yes to a degree, and that’s certainly good news for them. They’ve updated their apps, which get good reviews, but they still lack something that they simply are unable, or organizationally challenged to deliver. They haven’t been able to make clients like them. Some will say that this doesn’t matter. Clients will use whatever services incumbents give them and the “stickiness” of the relationship is such that a positive sentiment toward the institution is superfluous. That works when you’re the only game in town, but with 40+ neobanks actively pursuing your clients and ESG (environmental, social and governance) as a major societal theme it is doubtful if this is a good long term strategy. Perhaps incumbents should use some of their budget for the further digitization of their services and reallocate it to a redefinition of the client relationship?

That neobanks make jaded customers express positive sentiments toward their bank is remarkable.

Are neobanks set to disrupt incumbents? Yes, it is nothing short of remarkable that neobanks have been able to make jaded customers express such positive sentiments toward their banks. This does not, however, imply that this disruption will happen quickly, or result in the demise of incumbents. Neobanks, and their investors footing the bill, have to be in this for the long haul. Those touting the disruptive capacity of neobanks have reason to be encouraged that they are causing something transformational to happen in this sector.

How quickly can neobanks spread the joy?

Five years ago neobanks were barely a blip on the radar, today billions in brand value has evaporated from the traditional suspects, and some neobanks have established themselves as real alternatives in the eyes of consumers. The relation they have with their customers is clearly more positive than that of incumbents and I suspect that this will only grow given their rapid-fire roll out of new services. The question remains: how quickly can they spread the joy? The process of convincing people to leave incumbents will be slow, but the clients they win today won’t look back and as they spread enthusiasm to others I predict many more will make the switch.

Thanks for reading, please give this article a "like" and leave a comment to foster discussion. If you enjoyed this article connect with me on LinkedIn for more.

Rich Turrin is the international best-selling author of “Innovation Lab Excellence: Digital Transformation from Within” and an award-winning executive with more than 20 years of experience in banking and fintech innovation. He is an independent fintech and AI consultant living in Shanghai and helps clients navigate the uncharted waters of doing business in China. He previously headed fintech for IBM Cognitive Studios Singapore (IBM’s Innovation Lab) and worked for IBM China where he led his team to win the prestigious “Risk Technology Product of the Year” award for his unique hybrid-cloud solution to risk analytics. His next book “China’s Digital Currency Revolution: Profit from Banking Innovation that Will Shape our Future” is due out in June 2020. Learn more: RichTurrin.com.

Innovation Lab Excellence on Amazon

China's Digital Currency Revolution will publish in May. Sign-up for notification

References:

https://www.dhirubhai.net/posts/turrin_the-brand-values-of-eight-of-the-uks-biggest-activity-6578516713418133504-rPBc

https://natural-language-understanding-demo.ng.bluemix.net/

https://lifehacker.com/why-you-shouldnt-trust-app-store-reviews-and-what-to-1515379780

https://www.apptentive.com/blog/2016/06/23/mobile-ratings-good-bad-ugly/

https://www.forbes.com/sites/ronshevlin/2019/09/25/the-hidden-agendas-behind-apple-and-google-app-store-reviews/#65ee320746c2

https://thefinanser.com/2018/07/need-define-not-challenger-bank.html/

https://medium.com/@brettking/banks-just-arent-that-innovative-1bddae97eeef 

https://thefinancialbrand.com/80002/challenger-banks-legacy-institutions-mobile-banking/

https://www.businessinsider.com/uk-legacy-bank-values-decline-neobanks-could-be-to-blame-2019-9