Data Science and Game Difficulty Tuning: a New Paradigm for Mobile Game Studios!

Our studio has spent a lot of time in the past trying to tune the difficulty of the stages in our games, without really knowing what was the real effect on our players. So we decided to do something about that, not just for us, but for all mobile game studios and publishers.

In this article, we explain how a key process in game design -- difficulty tuning -- is being disrupted by Data Science and Machine learning, and how game companies that embrace this new paradigm will get an unfair advantage in player retention and monetization.

Player retention is key to the monetization of mobile games 

In the mobile game world, and especially in the casual and hyper-casual games market, all the economic models - freemium with IAP (In-App Purchases), Ads, subscription, even premium - rely on player retention. You know the magic formulae: you must have CPI (Cost Per Install) should be lower than LTV (Life Time Value of a player). 

You need to lower your CPI, and increase your LTV. And the more players stay in your game, the more they can watch ads, the more they are inclined to buy an IAP, the longer their stay subscribed and the more monthly revenues you receive from the subscription platform, and even for premium games, if players stay longer in your game, it means that they enjoy it, they will tell other people and buy your next game !

?? “But it’s not that simple, I know !”

Player retention remains the big challenge in mobile games 

At the last Pocket Gamers Connect in London (January 2020), during the hyper-casual track, several speakers explained how they have numerous tools and specialists to help them acquire players, but retention remains the big challenge. 

According to GameAnalytics (H1-2019-Mobile-Gaming-Benchmarks), you should target at least 35% for Day 1 retention (50% for Hyper-casual games), 11% for Day 7 retention, and a 4% Day 28 retention is considered good….. From all those difficult to acquire players, almost 90% will stop playing after 7 days ? 

?? “So why is it so difficult to reach a good player retention ?”

Difficulty tuning is key to player retention

Most game designers know the “Cognitive Flow” concept: each game has a specific difficulty progression, and each player will react to the difficulty based on their abilities, but also according to how they feel, like and enjoy the game. 

Players will churn (definitively stop playing the game with no coming back) for 2 main reasons: frustration (they have been frustrated too many times, and don’t feel like playing anymore), or boredom (they have been bored too many times, and don’t feel like playing anymore).

In casual and hyper casual games the difficulty progression can be quite flat (level 1200 of Candy Crush is not that harder than level 20 !-). So studios put some ? Pain points ? at some specific stages… not knowing really if they are at the right places for the players… which are all différent!

?? “Yes that’s right, but it’s so difficult and tedious to tune the difficulty of all stages!”

First challenge, studios are blindfolded when tuning game difficulty!

Testing the difficulty of all the stages in a game can be very time consuming. And you rely on feedback from your team, some testers… are they representative of your real players?

Then you put an analytic tool in your game, you set up events before each stage, you set up a stage funnel in the Analytics web portal…. and you see that you lose many players after level 10… so what ? Is it because it’s too difficult ? Too easy ? 

You change the speed of the opponent, you put 5 more moves in a Match3 stage, it should be easier for players… but is it really ?

And more importantly, how do you know that, for each stage in your game, you have a difficulty which is optimum for your overall audience of players ? 

?? “So how can I find, for each stage, the optimum difficulty for the overall audience of players ?”

The solution: Data-Driven Difficulty Analysis!

If you get data about players starting a stage, how they finish it (wining, losing, quitting…), you can get very important information about how your real players behave in your game, and how they react to changes in the difficulty while tuning your game. You can use statistical tools (like Gaussian regression) to work on the tuning of all your stages.

The data-driven analysis and decision-making can be done manually either by developers or by someone trained to get the most information from the data - an analyst or a data scientist. However, doing this manually could take a lot of time that studios do not have. 

That is why we developed askblu.ai, to provide this important feature to all mobile game studios and publishers with no investment costs (full SaaS solution).

?? “Now my stages have a difficulty level optimum for my audience…. that’s it?”

Second challenge: players are very diverse in casual games

We know that retention depends heavily on difficulty tuning; a frustrated player will churn (too difficult), a bored player will churn too (too easy).

But players in casual and hyper-casual games are very diverse!

If your game delivers the same stage difficulty to all players, you will lose players that could stay in your game if you could detect that they become frustrated (set it easier) or bored (set it harder).  

Therefore dynamically tuning the game difficulty becomes an unavoidable need for the game-designers and the developers. The assumed goal is to give to players the most comfortable experience, avoiding being frustrated if the game becomes too hard or bored if no real challenge exists in the game.

?? “I try to dynamically change the difficulty in my games… is that not a good idea?”

Dynamic tuning is not player personalization!

During the last decade, game studios have tried to dynamically tune the difficulty using mostly arbitrary rules, hardcoded in the game. Those rules, as simple as “the stage should be easier if the player just lost three times” for instance, has the advantage of being easily readable and having a simple design. However, a significant drawback is the lack of feedback on the benefits of such a rule:

       - Is the rule optimal? for all players ? 

       - What is the real impact on the player’s perception?

       - Does this really have an effect on the player’s experience and thus on retention?

But more importantly: you don’t really personalize because the rule is the same for all players. 

Why three levels failed rather than four or two? Some players might prefer trying more than 3 times and some other players won’t like to lose 3 times in a row...

?? “So let’s drop these “in game” rules…. what is the solution for real dynamic tuning?”

The solution: Real-Time Player Personalization with Machine Learning

Analyzing data and dynamically picking the best option can be automated, using machine learning, statistics, automation processes and cloud computing. This also allows us to take into account more complex information, with the price to pay being the readability of the decisions taken, for which more expertise is required. 

Decision trees, logistic regression, clustering, neural networks and statistics are common tools that can be used for such automation, where the detection of criteria that maximize a figure of merit - like player retention - is performed automatically, without the need to dig by yourself through the data.  

Our platform askblu.ai provides this real-time player personalization to all mobile game studios and publishers, with no upfront investment. Each game has its own Machine Learning model, because every game is unique (how people play it, when, how long…). 

?? “Thanks, that is exactly what I need, I want to try it now!”

Conclusion: Data Science and Machine Learning: a change of paradigm for game difficulty tuning

The player retention problem can be radically reduced, with a change of paradigm in the game difficulty tuning process. Rather than tuning a game blindfolded, and trying to program some “expert” rules, game developers will get significantly better results using a data-driven approach.

Taking again the example of frustration detection and experience correction, rather than considering making the stage easier after three defeats, a data-driven technique would allow you to discover that some players tends to churn after having lost only two levels, some other players would stay regardless of the result, and other players tend to churn once they get an easier version of the level after having lost that same level several times. 

This illustrates the variability of players, asking for a personalized experience that cannot be handled, or at most imperfectly, by arbitrary rules, but by Machine Learning algorithms that learn from the behaviour of your real players.

Want to try it ? Sign up your studio at askblu.ai or request a visio presentation (message me).