BEM Gamification: improving research

We've recently created a research group on Gamification, not only to improve our framework, but also to understand the state of the art of academic research on the topic. Any superficial search for bibliography will yield the importance of the field, specially on the topic of education, which accounts for about 45% of the research material according to Albertazzi et al. (2019), and the number of meta-analyses you can find are impressive. But our first findings created a deep worry on how academia has approached this research. Here I lay some points that I think could be improved so we can advance the field further and farther.

The emphasis on game components or elements

I understand that Gamification was born of thinking about the application of game elements like points, badges or leaderboards, or meta-mechanics -like levels or missions- on serious contexts, but it amazes me that almost all experimental research is still based on finding the effects of these components on motivation, performance and engagement. Albertazzi's "A Wide View on Gamification" meta-analysis is brilliant in searching the most used words in the field over time, and here it is:

The thing is, this shows that Gamification is still thought in terms of adding elements to contexts and see the effect on human behavior and motivation. The thing is, every experimental design not just focuses on components but also seems reluctant to alter the basic design: a call to action which gives points, that affects a leaderboard and which can unlock badges and levels. It is like if culinary research studied the ingredients but was reluctant to find the effects of the recipe! Like finding the effect of ketchup in pasta, rice, brocolli and so on. This of course creates easy-to-measure results, but has created conflicting results as well, because the design of the experiment has a large impact on the outcomes.

One study (Mekler et al., 2017) tried to find the effects of intrinsic motivation on a system where people would be rewarded points, badges, leaderboards and levels for creating tags on pictures. The results were unsurprising (not an effect on intrinsic motivation), as the task was a reflexive one, unchallenging and based on the basic extrinsic-based reward-cycle system. If you are searching for the effects on intrinsic motivation, you need to design better experiments, where the game design should push towards mastery or autonomy at least, but here there was no meaningful choices or a real challenge.

Where are the theories on Feedback and Information Flow?

Gabe Zichermann once said that Gamification was 80% psychology and 20% technology. Not only that phrase is absurd as it is trying to compare different categories that are not mutually exclusive, but also created an imperative where the basis of gamification research should be based on human development and behavioral sciences frameworks. Since 2015, every Gamification research quotes Deci et Ryan, or people from the school of self-determination theory. I'm not against that, but we have been looking the field from a narrow spectra of theories, and one thing lacking from many of those studies is how human behavior is modified but information flow, complexity, probabilistic calculations and so on. Again, Albertazzi's study (which was the trigger for this article) shows that while 45% of research is based on education, only 4% is based on information sciences. The fact that "feedback" or "call-to-action" or "notification" are not among the most recurring words on the research (but "rewards" is), tells me there is a deeply rooted conceptual problem. Not even "game" is among those words, and game design has a very strong element of information design. Mekler's study even made the decision of not creating a good interface for their experiment, to avoid tainting the results, but the interface is part of what should be thought of. Affordances regulate behavior as much as "points".

Where are the game designers?

In the early days, again, I remember in GSummit 2013 someone from Bunchball said that Gamification required behavioral scientists, not game designers, which of course I disagree with profoundly. Talking about Game Thinking if you have never thought about game design, is just, again, psychological research based on game components, but not game paradigms.

Experimental designs are being done by people that probably could not understand how positive feedback loops can ruin games like Monopoly but can make amazing games like Final Fantasy. Researchers have not a clear distinction on what a game mechanic is, how a black box works, and how rules affect behavior optimization. My research group has 4 psychologists, only two of them are gamers and only one has tried to design games, so I have placed upon them the homework of watching everything Mark Brown has placed on his youtube channel. Of course, they will have to read Zimmerman and Schell and Callois and McGonicall and and Lazzaro and so on, but an initial understanding on how game analysis is conducted might be enough to be able to push research towards other realms.

Think on how every leaderboard in gamification is based on behavior-rewards cycles, which a bit of reading on human motivation theory should alert you towards not doing. So I ask you, how do sports leaderboards work? You do not get points in soccer for "kicking the ball", you get them if you manage to make the ball cross a line on a give space. And the leaderboard is not based on goals, but on matches. The abstraction has several levels, to keep the players engaged on the problem, not in the points. So why we keep awarding points for repeatable, mechanical behaviors?

But I see few references to game designers or mentions to game design paradigms applicable to gamification. Granted, there is not a lot of research being done by game designers, being a practical, non-academic discipline by tradition (which should change, why is there graphic design, industrial design, textil design majors and no game design majors!), but there are amazing studies like Cowley, Charlie et al. "Towards and Understanding of Flow in Video Games", that could teach us more about gamification design than studies on the effects of points on intrinsic motivation (we already know plenty on this topic).

What about choice architecture?

Some articles actually mention meaningful choices in their studies (Albertazzi's gives a mention as one important concept discussed in the field, for example), but the definition and approach of experimental design based on this concept is still lacking. Barata, Gama et al. (2016) made an experimental approach which included sorts of technology trees, which had a positive impact on the learner's performance. But, beyond giving students the choice of deliverables for different amount of points, little has been done in this regard. Choice architecture is a big field and modulates much of our engagement as players: there are risk taking choices, incomplete information choices, probabilistic choices, time-pressured choices, delimited and unlimited choices, strategic and tactical choices, illusory choices, long-term repercussion choices, etc. Gamification will not survive if it becomes only a skinner box for nudging behaviors, and the way to avoid that is finding strategies that create choice or illusion of choice at the very least. This could be a field of study on its own.

Game mechanic-based and design-based research

Although the feedback elements that are studied are few (I work with a repository of over 80 game feedback elements in my designs, including audio discriminatory signals), studies on the research of game mechanics is almost non existent. This stems again from the fact that gamification bought one game "schema" of design and has tried to use it again and again in different contexts (lazy design). But what about including some game mechanics to the mix, without creating full-fledge games. The distinction of serious games and gamification has damaged this line of research, as researches avoid crossing the line towards game-based learning or the such. But if you apply game mechanics to real world behaviors, you are not necessarily creating serious games. Games require a delimited game space and time (the magic circle), gamification does not, and rule setting does not create that kind of scenarios as a mandatory effect. So what about understanding investment mechanics, deck building mechanics, worker placement mechanics, character building mechanics (different kinds), pick and delivery mechanics, and so on?

This leads me to a second point: research on the design process itself. You can see that this has been the work of framework designers (read Mora, 2107, "Gamification: a systematic review"), but think in how the corpus of theoretical research is built in fields like architecture, industrial design and so on. They study the effect of particular design paradigms on particular contexts with particular users. My sister, Ana María Velasquez (co-founder of our framework, and PhD on Human Development) delivers a course on civics and peace at the University and has been reluctant to gamify it, because points, badges and leaderboards will not cut it, as the course is about introspection and discovery. We are looking towards other game paradigms, like how games deal with morality, or how ideogames work on self-discovery (read Dave Eng's article on the topic on XP University for more info). That could become a topic of research!

There is more to think about how the field should tackle its own research in the future. Albertazzi et al. end by predicting that practical studies will overcome empirical or theoretical ones, but I believe this trend will only become a series of case studies that will not allow to create a corpus of paradigms and schools of thought. But they also conclude that more multidisciplinary studies will appear more and more, and I'm eager to read them, because that's what we need: to stop looking at our own bellies and start searching beyond!