Why traditional assessment providers are frightened of Game-Based Assessments

As market interest and adoption of gamification in recruitment is ramping up, so a number of traditional assessment providers are rushing out blogs and white papers on why game-based assessments are not scientific and couldn’t possibly add a new and powerful way to identify someone’s suitability for a role. Yet, their analysis of gamification in recruitment sadly stems from ignorance (and increasingly fear) of how game technology can and has been successfully applied to the world of psychometric measurement. Arctic Shores is a global provider of game-based assessments (GBAs) and using the results from over 50 live recruitment implementations and 150,000 completed assessments, we have comfortably demonstrated that game-based assessments are effective.

The first and most significant claim by these traditional providers is that game-based assessments are ’games’ and therefore unsuitable for the high-stakes environment of recruitment. If a game-based assessment’s foundation was entertainment, then indeed it’s application would be questionable - not because turning a boring activity into a fun one is bad, rather it creates a conflicting message to the user on whether to treat the task as high-stakes or low-stakes. The real fallacy of this argument, though, is that it fundamentally misunderstands the application of game technology to psychometric assessment. Our game-based assessments are not based on entertainment, rather previous and validated neuroscientific and psychometric research. What we and other game-based assessment providers have done is taken that research and applied a modern and mobile native user interface. We don’t use the word ‘game’ or fun in any of our communication to the user.

The reason that some traditional providers keep trying to link game-based assessments with games is because the disruption  game-technology enables through the collection of a much richer data set, one based on micro-behaviours – on average 5,000 data points per 35 minute assessment session. This is 100 times more than a traditional assessment (based on a 50 question assessment and many are in a race to reduce this number) – no wonder some of the traditional providers are so alarmed.

The second claim, a derivation of the first, is that any platform that uses game technology must both give an advantage to ‘gamers’ and introduce a practice effect. Again an assumption that fails to appreciate the complexity in the design of the psychometric tasks in a game-based assessment. While an entertainment game is built on acquiring a skill or knowledge of a mechanic, achieved through repetition, “proficiency” is not part of the measurement method in a GBA and that the elements in GBAs that are prone to improvement or dependent on exposure are completely irrelevant to the psychometric framework underpinning the tasks. What we measure are the behavioural markers that are extracted from our careful replications of cognitive paradigms, which are not susceptible to an exposure effect. Furthermore, we have run validation studies to prove that game experience has no impact on our results.  The advantage of using game technology in the design of game-based assessments is the tracking of subtle changes in behaviour across multiple tasks. So the measures of personality are derived from a large and cross referenced data set.

The final claim is the myth that the best and only use of gamification in recruitment is to ‘gamify’ a traditional assessment. This approach involves taking a traditional aptitude test and applying a game-like backdrop and context to it. While this is an approach to gamification in recruitment, to suggest this is the only viable approach is demeaning to both the user (gamification should only be used to make a boring test seem less boring) and the many companies that are striving to address the adverse impact amongst certain groups in society created by an aptitude centric screening process. Take Siemens in the UK, for example, where using a traditional set of aptitude tests to screen candidates resulted in a 16% female job offer rate, which improved to 32% when both a game-based assessment and a gamified aptitude test were used. At the same time, the number of candidates Siemens UK needed to see in order to make a job offer dropped from 5:1 to nearly 2:1. These aren’t small improvements, they are game-changing and we are seeing them repeated in different companies in different countries.

The traditional providers haven’t made a game-based assessment so all their hypotheses about how a game-based assessment might work and the issues that might be encountered are in the end nothing more than speculation. This is what happens when any breakthrough innovation is introduced to a market – James Dyson saw it when he introduced a new vacuum-cleaner technology twenty-five years ago. In the end the market will determine if an innovation is truly disruptive and no amount of fear mongering from the entrenched operators will change that. So when a traditional assessment provider tells you that there is no future for game-based assessments ask them what evidence they have to show that measuring personality through behaviours and cognitive responses is not possible.

As a small, fast-growing company, we are flattered to be getting so much attention from some of the traditional providers, but we are not willing to sit back while the science behind our innovative approach to assessments is hijacked and misrepresented. As my favourite philosopher, David Hume, said: “A wise man proportions his belief to the evidence”. We have plenty of evidence and science behind our game-based assessments and a large and growing list of companies who have taken the time to review that evidence objectively and verify it in their own processes. Everyone has a choice - don’t let someone else’s fear of the new drive yours.