Psychometric Test Completion Rates: A Vanity Metric That Doesn't Matter

Hiring managers, talent acquisition specialists, and even occupational psychologists very often get the cart before the horse when it comes to statistics. People (myself included) can get very attached to certain numbers, fixating on their implications, and nowhere is this more apparent than in psychometric testing.

When adopting psychometrics for the first time, the biggest concern of many organisations is the completion rate. Far too many organisations are hesitant to adopt them in the first place for fear of candidate attrition, which is an incredibly myopic perspective on selection.

In reality, completion rates for online psychometric tests are, at best, a vanity statistic that serves only to provide reassurance to nervous talent acquisition teams. It really isn’t the be-all and end-all that people think, and it certainly shouldn’t be a show-stopper to designing an effective selection process.

Total Completions > Completion Rate

ROI from an employee selection process (aside from cost) is determined by these three variables:

Assessment validity (R) × Value of performance (￡) × How selectively you hire (Z) = R × ￡ × Z (Schmidt & Hunter, 1998).

The ideal situation is one with a powerful predictor of job performance (cognitive assessments, structured interviews, etc.), a role where high performance really matters (professional, technical, managerial work), and very high pass marks for success (top 10%, top 1%, etc.).

High pass marks require large samples, because screening out 99% of your applicants requires a minimum of 100 applicants; otherwise, you screen out everyone. Moreover, the larger the sample, the more exceptional candidates you can identify, meaning that the total number of completions is directly proportional to the value you get from hiring.

As a result, the only reason completion rate matters is because it is related to the total number of completions, which is orders of magnitude more important.

Consider this: imagine you have a graduate scheme, and you invite 1,000 candidates to complete an assessment, but only 50% actually do. You now have 500 applicants to work with, allowing you to be super selective and only progress the best.

Now imagine instead that you aggressively CV-screened out 990 candidates and only tested a subsequent 10 applicants, with a 90% completion rate. Yes, 90% is higher than 50%, but 500 is much higher than 9, massively limiting the value you get from assessments (not to mention making the recruitment process infinitely less scalable and efficient).

Given that CV screening is a gigantic waste of time for early career hiring, that means that hundreds of high performers would have been screened out needlessly. In the former group, although 500 candidates deselected themselves, you still have dozens of high-potential applicants in the pool, representing a huge opportunity for the employer.

This matters because many organisations structure their selection process like the example above in order to maximise completion rate at the expense of total completions, which is extremely backward logic. Particularly for early careers hiring, assessments add by far the most value in the early stage of the recruitment process, and the later you use them, the less value you get irrespective of completion rate.

Naturally, not all roles have a sea of potential applicants, and sometimes the only way to boost total completions is to maximise the completion rate, but that still highlights why total completions are the metric to watch rather than the completion rate.

The Benefits of Candidate Self-Deselection

Many (perhaps most) organisations that use psychometric tests do so on a pay-per-use basis, buying individual tests or paying per candidate. This matters because the costs of using psychometric assessments are a major factor in determining ROI, and HR budgets are stretched thin enough as it is.

Given this reality, do you really want every candidate to complete the assessments?

Surely, you want non-serious applicants to back out of the process without deducting valuable assessment credits on your platform. Additionally, collecting this information comes with solemn responsibilities for data protection, which can be avoided if they just don’t waste your time.

This is particularly true now that candidates are using ChatGPT and other generative AI to write application forms and CVs for them. Consequently, a great many candidates are not truly invested in the recruitment process, nor have they seriously considered actually working for your company. Instead, they are following a scattergun approach, with the expectation of ignoring most organisations that get back to them.

Perhaps more cynically, some candidates may be making fake applications in order to test the effectiveness of their CVs, seeing which variants yield the best results so they can optimise their real CV. AI truly is changing the face of recruitment, and unfortunately, it seems to be making the lives of HR professionals harder.

Psychometric assessments are therefore a great way of determining who is serious and who isn’t, representing an advantage rather than a disadvantage. We also know from the research that low performers are the most likely to drop out, not high performers (Hardy et al.).

As the saying goes, sometimes the trash takes itself out, and that is a good thing.

Candidate Experience ≠ Completion Rate

Many organisations use completion rates as a proxy for candidate experience. After all, if people don’t like the test, surely the natural reaction would be to drop out? In reality, however, this relationship is far from perfect.

This is particularly true when looking at the raw completion rate—i.e., the number of completions divided by the number of invited candidates. The vast majority of those who drop out do so without ever opening the assessment, having no bearing on the perception of the test itself.

Instead, of those who do start, calculating the percentage who then finish would represent a better indicator of candidate experience with regard to the test. However, the total completion rate is influenced by far too many factors to pull specific insights about candidate experience.

Moreover, a high completion rate could be masking issues with candidate experience. In certain industries, completion rates are disproportionately high at baseline, giving a false indication that the chosen assessments offer a great experience. In reality, however, the candidates may just be unusually desperate and will suffer through anything to progress to the next stage, obfuscating the problem.

Instead, capturing actual candidate experience statistics would create a more accurate picture—i.e., NPS and satisfaction surveys. These could be conveniently sent to candidates during or immediately after the selection process, giving actionable information instead of questionable inferences. This is the logic that underpins the use of psychometric testing in general: one should always directly measure important characteristics rather than infer them from some secondary variable—i.e., measure cognitive ability instead of screening using academics.

Summary and Practical Implications

High completion rates for psychometric tests look good, sound good, and provide superficial reassurance to nervous talent acquisition specialists without actually mattering in practice, making it a clear vanity metric. I understand why people fixate on them, though, as many simply don’t know what else they should be focusing on.

In reality, the total number of completions is what actually matters, and completion rate only matters with respect to this. If you are flooded with applicants and only 50% complete, you are almost certainly in a strong position to be highly selective. Naturally, if you can get your completion rates up without reducing the size of your applicant pool, then you get a larger total applicant pool, which was always the goal.

Instead of looking at completion rate as a quality control statistic, you should just see it as one of many levers you can pull to get more completions. If there is no way you can expand your applicant pool, carefully managing your existing candidates is your only hope, which means optimising completion rates.

For the vast majority of organisations, however, removing unnecessary requirements for application, using assessments earlier in the process, and reducing reliance on CV sifting can substantially increase the number of people tested—far more than optimising completion rates.