BEM Gamification: Measuring Motivation (part 2)

To read the first part of this article, click on the link above.

In the last article we discussed behavior clusters, and measuring energy and direction. However, I left the most important indicator out: perseverance. So let's pick where we left off!

Perseverance

While energy can help us understand how attractive is the activity we are designing and how much intrinsic motivation is being allocated to the activity (or the measure of fear for a consequence, but let's say we are not that kind of designers), perseverance shows us how this energy is modulated over time.

Motivation is not a static indicator: its rate can change, and will change over time. Motivation is in flux, and it depends on several factors, many of which are out of our control. Motivating someone is an uphill battle that can be negatively affected by several foreseeable and unforeseen forces. Let's list a few:

1. Workload: You might have a game that you play every day and love, but a peak in you workload makes you leave the game for a few days. I have seen this peaks in several gamification projects and, actually, if the tension created by this workload lasts several weeks, it can produce permanent disengagement. As games usually are voluntary in nature, they are affected hard by non-voluntary activities.
2. Repetition: Our brain is built for novelty, an we know for a fact that repetition affects the amount of dopamine that is released upon anticipation. Games are repetitive by nature, as are built upon cycles, but, the game designers work hard on keeping the game fresh by changing the level design. How many songs can you play with the 88 keys of a piano? How many novel problems can you create with 7 game mechanics? Actually a lot, but when the brain picks on patterns it begins to feel satiation, and that is why series with more than 5 or 6 seasons are rare.
3. Frustration and Boredom: We can never lose ourselves from Flow theory! Humans have the ability to withstand variable levels of frustration and boredom for some time, but sustain this feelings and this is a recipe for avoidance. The issue, like with repetition, is that keeping the player in the flow channel becomes increasingly hard, as you have to accommodate for the real fact that your players actually learn from your system! What makes games powerful is what makes them "finite" in a sense. As skill progresses it becomes harder to keep the player engaged, specially because the more skillful a person feels she is, the less inclined she will be to expose herself to challenging situations.
4. Replacement: We have limited time in this world, but, even worst, we have limited time per day! Humans work in circadian cycles, creating structures of behaviors and habits over 15 to 18 hour cycles. This means that most of our daily "calendars" are filled out by serious work and idle activities. Even when work becomes disengaging, people will find other activities to fill those spots, like social media or games. This means that, if you get to be part of that calendar with your design, your slot will be constantly bombarded by hundreds of other potential activities. You might lose a player only because she found something more interesting to do. Or your player might be interested in your system, but never finds another slot to replace.
5. Pain and Effort: I said that the more skillful a person is, the less inclined to expose herself to challenging situations. Let's tackle that issue now! Most activities you engage with will come with a degree of physical or psychological pain, and will ask you to make an effort to continue. While many product designers are trying to create engagement by reducing pain and effort, game designers are not in that business! Mastery requires that you put yourself in uncomfortable situations. Go to the gym and survive the body aches, learn to play guitar and survive the pain in the fingers, learn a game and prepare to lose over and over again. People tend to endure these hardships until they feel adequate enough: learn to drive a car and you will stop learning when you can drive without thinking, but you will not expose yourself to more challenging situations. In this sense, it is normal for players to abandon before reaching the end of your game, just because they don't feel like exposing themselves to higher challenges.
6. Skill Decline: We live in a world were certain skills peak on a certain age, and then both physical and cognitive functions naturally decline. Professional sport and e-sport players tend to be competitive until they are 30 to 40 years old, as younger brains and bodies are more effective by nature. This is not something to worry unless you are thinking on having a player in the very long term, but is something to take into account if you are pitting players of different age ranges, as the difference in skill can make some players abandon the system quicker.
7. Serotonin and mood changes: To talk about motivation on a healthy brain is hard enough, but when you have to take into account how depression or mood variance due to environmental changes affect you potential players, you are in a whole other level of challenge. Depression for example creates more feelings of anxiousness when you are put into challenging situations, which means you might create a really engaging system that actually creates avoidance behaviors for some people. And this can happen suddenly: you might have engaged players that you then lose without apparent reason.
8. Prediction: When a system becomes too predictable, it again loses novelty. To avoid this, game designers tend to give games a lot of volatility, which is easier to do in commercial game designs than in more serious projects. How to introduce volatility if you are trying to teach a procedural system, where the idea is that players become able to predict how the system will react?

This is a small list to just show how many of the elements that diminish perseverance are out of your control, and others require careful thought and testing. Think about this: the triple A game industry can take 3 years and big teams to keep you engaged for 50 to 70 hours, and even then, you will reach only a fraction of the population. Measuring the life time of your players become essential to understand motivation and, while you might never know the real cause of abandon, understanding when your players are leaving can give you hints into the problems that are in your control, like difficulty curves or fatigue from repetition.

The game of motivation

Motivation designers of any type are fighting a hard fight, with many obstacles that play against us. We can't control all the factors, which means the game of motivation is a game of both skill and chance! The right question is about the odds of getting a player to try your system and then the odds of keeping them for a significant amount of time.

If you are designing a mandatory course, you will not be able to measure things related to intrinsic motivation, so you will have to measure the perception of the experience. But this is relatively simple and has been done through many methods. But when you don't intend to create obligation, now you are playing a hard hand. Specially because in serious environments you don't have a lot of space for testing, and, as an expert, your client might expect certain results in very short timeframes. In this case, measuring motivations is about managing expectations!

Unfortunately, gamification companies and consultants have always used a really optimistic language like: "gamification is the perfect tool for engaging your users/learners". It's understandable, but when a client feels like gamification is THE answer, this might create the idea that gamification is the sole responsible for the success of a project and that it will work with everyone! And the measure of success is a game of expectation and anticipation.

As a game of probabilities, measuring motivation in percents seems like a good move, but you must be careful, as scales affect expectation in a big way. Look at the following questions and answers (not taken from real stats):

1. What is the probability that my current users will be engaged by the gamification system? => 80%.
2. What are the odds that someone that finds my site will be engaged by the gamification system? => 60%.
3. What are the odds that my users will invite new users to the experience? => 8%.
4. What percentage of my possible market will engage with my platform? => 3%.
5. What percentage of people in the world will interact with my gamified app? => 0.001%.

You must be wary as many of these questions won't measure motivation via gamification, as, for example, attracting new market might be the role of the marketing team (which can be using or not gamified principles). On the other hand, percentages can give you odds, but can also give you sets: 3% of engaged users of my possible market is not the same as having a 3% probability of engaging a user. And finally, the framing of the question can take you to the bottom of the scale, which sounds awful! Is 0.001% of the world population a bad metric? Sounds bad because of the scale but it's actually around 8 million people!

The thing is, motivation can only be measured in numbers 1 and 3! If you invite 100 people to do a task and 15% do it, that might be more or less the chances that that task can engage people. In this case, the set represents more or less the chances, because all of them had the information on the task (and it was a big enough sample), very different to say that 3% of my possible market engages, as 97% might just not know about your system! In this sense, motivation can only be measured by understanding the level of engagement of people who are still active in your system and receive the call to action!

If you have 1000 users and, after gamifying your app you get 1000 more, was this solely because of the gamified system? That is really hard to say and you would need to have control groups to have better information (having the non gamified option competing with the new one, for example), but also, eliminating noise can be hard! Maybe there was a better marketing scheme that is responsible for the new users. So, measuring motivation by growth is an inexact science. Bounce rate might be a better indicator, as it show the resistance to grow and can be expressed as probabilities: 60% of my new users engage for more than 10 minutes.

On the other hand, if with each new user you recalculate the odds of people doing certain tasks, this can improve your model! If you send a call to action to 200 users and 120 are not engaged, that would be the measure of apathy for that task. Is that a bad number? Not necessarily!

First of all, remember the behavioral clusters! The more activities you can do in an app, the more they are competing for the user attention, and one related task might be taking people's attention away from the other one. In this case, improving the engagement on the task might systemically reduce the engagement on other performing activities. Remember, you have limited amounts of attention!

But this shows one big thing! The only way to measure motivation is giving choice, as choice is the only way to measure chances! As player motivations can vary a lot among each other, having a mechanic which only 10% of players interact with it might not be a bad idea: maybe it is a competitive scenario for only high performers, but this 10% yield 90% of the productivity! Remember, you need to account for energy as well as direction. Now, if that 10% loses interest in just a few weeks then you have a problem of perseverance for that mechanic and should be tweaked. So measuring engagement and disengagement over time is a key metric!

And be wary: one thing is to lose a player, another thing is that a player stops using a mechanic! Maybe that 10% that stopped using your competitive scenario migrated to collaborative guilds! That might be a great player journey! In this sense, measuring engagement by player's level might give you great information on how your mechanics promote motivation. But this can only be achieved if your players have choice!

If your gamification system is a way to remove choice then you will have a hard time measuring motivation. You will have to rely on past baselines to see if your current implementation improve the odds of engagement. But you will have to isolate variables like marketing and this would require the type of control testing that is hard to do in the corporate world. Think of putting your own mechanics in competition and you might find a better way to check for results, but always remember, check the energy, the direction and the perseverance to have a bigger picture.

And never oversell expectations! Remember, there are 3.68 billion gamers in a world of almost 8 billion people, which is less than 50%, and World of Warcraft has 4.8 million subscribers, which is 0,13% of that 50%. Now you can start to really understand what getting another 100 players to your system might mean! You gamification system is competing with all sort of mandatory and idle activities, some of them with enormous budgets, so any new user with feedback of love to your system is an epic win!

Any questions? I still have many! Any other ideas? Let's build knowledge together!