Your Leaderboards Probably Won't Work: Get it Right or Skip Them

Among the most recognizable forms of gamification, leaderboards are nonetheless often misused and can have a demotivating effect if poorly implemented or implemented in an environment where they are not appropriate. I've used them quite a bit and seen very few truly motivating examples that will drive player behavior. Mostly they come across as a waste of resources that can turn off a lot of players which makes them better to simply skip.

In this article, I'm going to summarize the top reasons "you're doing leaderboards wrong" and go over what I would consider a best-in-class implementation in Duolingo.

Primer: How Leaderboards Motivate

I've written before (and I will again) about Self-determination Theory which highlights three pillars of intrinsic motivation: competence, autonomy, and relatedness. A basic leaderboard can satisfy the needs to feel both competent and related.

Initially proposed by Leon Festinger in 1954, Social Comparison Theory centers on the belief that there is a drive within individuals to evaluate themselves. This is most true in situations where there is no ability to self-evaluate. Consider that if someone scores 3,000 points, they still don’t know if that’s a good score. They may gain some insight from their previous plays, but they don’t know if that is good in the grand scheme of things. Knowing how others have performed helps satisfy the need to evaluate oneself.?

Advances in Social Comparison Theory have led to explorations of related drivers of self-improvement (once I know how I stand, I next need to get better) and self-enhancement (I want to feel good about where I stand).

While these dynamics work well for those at the top of the leaderboard, they often demotivate everyone else. Thankfully, there are some techniques you can use to alleviate this inequality. Let's walk through the common problems with leaderboards and how you can fix them.

Your Leaderboard is Stagnant

It's 1978 and you walk into an arcade and play some Space Invaders. You feel like you did pretty well but then you see something you haven't seen in a game before: a leaderboard. You're nowhere near close to those scores. You play a few more times but you're not cracking that top 10 list any time soon. A month later, it's the same thing. Those top 10 scores aren't budging.

Today, leaderboards have evolved to support more than just three initials and can store more than the top 10 games (a single skilled player can occupy all the slots themselves), this experience is all too common in leaderboard design. The only people being motivated are those guys on the top. For everyone else, it's kind of a bummer and can make you feel worse about how you played. Even the top performers may find little motivation to keep playing if they aren't in any danger of losing their positions.

You could ease this by using relative leaderboards which show only the scores close to the player as opposed to absolute leaderboards which show the top scores. This allows the player to see progress as they improve (competence) and focuses on helping the player compare themselves to others (self-evaluation). But that rank 802,924 is still not much of an inspiration.

A better approach—and something I consider a must-have for good leaderboard design—is resetting the leaderboard periodically. Getting a top score is great but you're only holding onto it for a week (or however long it takes to reset them; I usually suggest a weekly or monthly cadence). This allows newer players to have a shot at the top even if it's only temporary. More importantly, it gives all players a reason to come back each week to claim their place on the board.

Resetting is especially important for cumulative leaderboards which measure an accumulation over time of what you're tracking. Without resetting, new players will never have a hope of climbing to the top no matter how skilled they are because others have been accumulating longer.

Your Leaderboard Isn't the Right Size

At a certain point, the rank on the leaderboard loses meaning when the number is too high. You're so far from the top and making so little progress at climbing that the leaderboard is meaningless or, worse, a turn-off.

At first, a player may make progress moving up as they gain competence in the game. This can satisfy drives to self-improve and give contextual feedback on how the player is doing against the general population. But over time you tend to settle into your best score and stagnate. The competition itself doesn't feel important because it's thousands of people you don't know anything about. You have nothing in common with them. They're just names of strangers. In fact, if all names on the leaderboard were bots, players would likely not notice the difference.

Arkadium experimented with a fictional leaderboard in which other participants were bots. The intent was not to deceive players but rather to cut development time in a prototype by implementing a simulated version. However, their scores would adjust over time to maximize the player's experience so they could always feel slightly challenged while still moving up often. In retrospect, this fake leaderboard, because it adjusted entirely around the player's behavior to keep them motivated, is better at influencing user behavior than most real implementations.

To resolve this, cohort players into smaller groups. Location, entry time, or even just random groups of 30-100 are better than a leaderboard of hundreds of thousands.

On the other hand, leaderboards that are too small with only one or two players can also be pointless. They won't likely create demotivation but they will also fail to add anything, not even the ability to self-evaluate. If you are resetting your leaderboards and you don't have a huge player population, you might find yourself with empty leaderboards.

One site had a clever solution: Sporcle, a trivia quiz site, has a "SporcleBot" which will show up on leaderboards with an average score allowing players to self-evaluate.

However, if you don't have enough players, skipping leaderboards entirely and just having a player compete against their own best score or other metrics can be simpler to implement and more effective.

Your Leaderboard isn't Ranking Appropriate Metrics

When considering adding leaderboards to a product, it's important to make sure you have something appropriate to rank your users on. If finding a suitable metric is difficult, that's an early indicator that leaderboards aren't a good fit for the product in the first place.

Similarly, the metric should be something that your users have control over and can actively improve on. For example, while fitness apps often show positive effects by adding leaderboards, you wouldn't want to rank users on weight lost because it's not something that users have as much control over (metabolism, starting weight, lifestyle, and genetics are big factors) and it's definitely not a good indicator of building healthy habits (ex: exercise will build muscle which can cause that weight number to go up). A better metric to motivate users would be steps taken or meals tracked which reward the behavior change instead of the results.

Another reason the ranking doesn't work is if the system too easily allows for cheating. Sometimes, it's not simple to avoid cheating in a system. Cheating can also be a sign of strong engagement so it's not always necessary to curtail. However, when the other players in a system can see the cheating, the whole leaderboard system can feel like a failure. This used to be very common in the mid-2010s when Google and iOS were pushing their platform leaderboards. Developers would add them to their games without adequate protections often where scores were being calculated client-side leaving them vulnerable. It wasn't uncommon to find most of the top of the leaderboards with scores of 999,999,999,999. In these cases, it would be better to avoid leaderboards altogether.

Also, consider that the metric you use will be the one you're driving participants to increase. If you create a leaderboard around something silly like lines of code committed for your dev team, you're going to encourage them to write longer code for no reason. If you rank your customer support reps by the number of customers helped you'll push them to sacrifice quality just to get their numbers up. Some settings are simply not appropriate to even attempt to drive competition. Apps geared around mental health can add pressure and feelings of inadequacy. Social media and engagement platforms can encourage players to game the system or simply emphasize popularity over quality. And really if you're ranking people who work together, don't encourage that competition related to their work. They're supposed to be cooperating not trying to one-up each other. Recognizing achievements is a far better way of motivating staff.

External competitions among coworkers that are unrelated to their work can foster camaraderie and enjoyment, however, as I'll expand on next.

Your Leaderboard Lacks Relatedness

Leaderboards work well when participants know each other. A weekly tournament at your office can work well because the participants have much more in common than just playing the game. It’s even better if you can change what’s being ranked week-to-week, giving more participants a chance to reach the top. These small settings work better because the players are highly related. They work together, they know each other, they see each other often, and can banter. This is where competition is effective because the competitors feel like real people.

Leaderboards in some systems such as Xbox Live allow for filtering by friends only which is a much better-related system. Another technique could be filtering by location (so I might only see other players located in NJ). Filtering helps with competence and self-enhancement by reducing the number of people being compared against. “I may never be able to be in the top ten of the world, but I may be able to make it to the top 10 in my city.”

The ability to zoom into other participants can also improve relatedness. Outside of leaderboards, studies have shown that people make more informed comparisons when considering other dimensions than the one under comparison (swimmers don’t just compare swim times, but also age, experience, and recent practice). These are “related attributes” to the main attribute being compared. These kinds of comparisons are easier to make when one is more familiar with competitors.

Best in Class Leaderboard: Duolingo

Duolingo has a cumulative leaderboard that resets at a weekly cadence. All the experience points (XP) a learner gains during the week are added to their ongoing score and players compete to earn the most XP. So you have a busy week and can't play as much, the next week you get to start fresh.

To ensure a level playing field and to keep the leaderboards from being too large, users are placed into cohorts of 30 formed by the time they enter. So everyone you're competing with has started at the same time and there's no one with an advantage of having started on Sunday night as opposed to Monday night. Thirty is a good number that is easy enough to scroll through the entire list without it feeling endless. This method of cohorting by entry time is common for cumulative leaderboards in many games.

Of course, there are still people who play more or play less simply because they have more or less time to devote to their language learning. To combat this, Duolingo also employs a league system where users can be promoted if they are in the top 10 on their leaderboard or demoted if they're in the bottom 10. This means that people who play less will end up staying in the lower leagues while high performers move up to the top. I've been in the Diamond League (the top of 10 leagues) most of the time I've played but I also spend over an hour most days playing.

Learners in the same league and cohort can actually compare profiles to each other! I can see what language competitors are studying, what achievements they have, how long they've been using Duolingo, and, most importantly, compare myself to them on a graph that shows daily XP progress through the week.

One more thing that's just a really sweet touch is the ability to add little emojis to your avatars on the leaderboard.

Another Great Alternative to Leaderboards: Crossclimb

I wasn't even going to include this originally but I had to call out a recent addition to LinkedIn's own Crossclimb game. While the leaderboards they use are cute since we're in teams, they're underwhelming especially because I don't have much control over the team , these cards you can swipe through at the end are a great alternative to leaderboards that satisfy many of the same motivations as leaderboards do. Card 1 comparing myself to the average of all players satisfies my desire to self-evaluate while the other cards satisfy my desire to self-enhance. "Smarter than 95% of CEOs" is certainly nice to hear even if it's just about how fast I played a word game.

Conclusion

• Always remember that leaderboards are fun for the folks at the top and much less so for the other 99% of participants.
• Reset your leaderboards regularly.
• Consider leagues or filtering to control the size.
• Allow participants to compare themselves on other factors besides the one used for the ranking
• Consider nixing the leaderboards entirely if you don't have many players, if the players have little to nothing in common, if you can't prevent cheating, or if you can't find a suitable ranking metric.

Select Sources

Festinger, Leon. “A Theory of Social Comparison Processes.” Human Relations 7, no. 2 (May 1954): 117–40. https://doi.org/10.1177/001872675400700202 .

Griffin, John. “Leaderboards - the Original and Best Social Feature ...” Gamasutra (blog), July 28, 2014. https://www.gamedeveloper.com/design/leaderboards---the-original-and-best-social-feature- .

Hollander, Adam. “Why The Gamification Trend Fails At Most Companies.” Fast Company, June 3, 2014. https://www.fastcompany.com/3031324/why-your-company-should-think-twice-about-gamification .

Jia, Yuan, Yikun Liu, Xing Yu, and Stephen Voida. “Designing Leaderboards for Gamification: Perceived Differences Based on User Ranking, Application Domain, and Personality Traits.” In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems? - CHI ’17, 1949–60. Denver, Colorado, USA: ACM Press, 2017. https://doi.org/10.1145/3025453.3025826 .

Ryan, Richard M, and Edward L Deci. “Self-Determination Theory and the Facilitation of Intrinsic Motivation, Social Development, and Well-Being.” American Psychologist, 2000, 67.

Wood, Joanne V. “Theory and Research Concerning Social Comparisons of Personal Attributes.” Psychological Bulletin 106, no. 2 (1989): 231–48.

Heather Arbiter is a Gamification/Game Designer and Product Manager who has designed leaderboards in countless games, platforms, and systems. In 2018 she did extensive research to create a deep dive into how and when leaderboards are effective to better guide clients mostly on why they shouldn't use them.

#FeatureFriday is a biweekly newsletter about the intersection of product, gamification, and behavior written with a personal touch. This is the 29th edition of the newsletter.