To Test or Not To Test: Validation in Question

With skills-based hiring on the rise, the obvious question arises, how to validate those skills in candidates?

Validation isn’t a new problem in hiring, employers have always sought ways to quickly and objectively assess candidate quality, particularly when hiring for multiple roles or when there are large numbers of applicants to screen for a single job.?

What is changing is what exactly we are attempting to validate.? In the past, the goal was to validate proxies for overall job fit, things like degrees, work history, and relevant backgrounds. The movement towards a skills-based approach challenges us to look for evidence of specific skills, whether or not they were acquired through formal education or direct job experience.

What we mean by validation

Something that is often overlooked in the discussion of validation is what exactly we mean by it.? It can be tempting to jump into the logistics.? Do we choose a binary or a threshold measure?? Do we use standardized assessments, social signals, AI, or a combination?? However, jumping to the how of validation before clearly addressing the what is why many of these evaluation methods miss the mark.

True validation involves 3 components.

1. A shared understanding of the meaning of the skill and rubric to define it.
2. Clear definitions of the level and context in which the skill will be used.
3. A mutually agreed objective process as an impartial judge of that skill.

Notice the objective process is not specified, because there is no single right way to perform an evaluation that works in every case.? The process will be shaped by the skill itself, the context in which it is used, and the stakes if the validation is not accurate.?

To illustrate, the risk of hiring someone to perform complex neurosurgery who claims, but doesn’t have surgical skills is very high.? The risk of hiring a marketer who claims to, but has never used mass-mailing software is relatively low.? Therefore the bar for evidence of these two skills will be different.??

Why shared understanding and agreement matters

Notice that each component requires communication, understanding, and acceptance.? If employers and job seekers have different understandings of a skill or how it will be applied in context, no evaluation process or third party will be able to provide accurate evaluations.

Arriving at these shared definitions and pre-agreed upon validation processes can seem daunting particularly when attempting to find a single standard to apply across the board.? For example, many people can claim to have used Microsoft Excel. However, there are varying degrees of expertise in Excel, including the use of macros, plugins, and pivot tables that would be necessary for a data analyst, but maybe not for an administrative role. This might lead to a candidate with basic knowledge of Excel feeling they are qualified for a role since they “have” the skill, and becoming frustrated when they get passed over. Employers will also be frustrated with the number of underqualified candidates they receive.

Complications in context, clarity, and common understanding mean no one has quite cracked the code for validating the requirements of multiple jobs at scale (although, some have come close for specific industries - see Scismic for Biotech Hires). Fortunately, we already have good, low-tech ways to address all three components at the scale of a single job.? The trick is doing them well, every time.

Validation Best Practices

The job description is an often underused tool you have for ensuring that all three components of validation are met.? Be sure that your job description includes enough information that a job seeker can understand the skills you need, in what context the skills will be used, and how you plan to validate them.

Requirements/Qualifications

This section is an opportunity to define the skills you need.? Rather than simply listing a skill, be clear on the specifics.? You could list, “PowerPoint”, but it’s much clearer to the jobseeker if you say, “ability to use themes, animations, timers, and notes to design compelling slides that convey complex topics using Microsoft PowerPoint”? The person now understands that they want someone familiar with all of the features of PowerPoint, rather than just the basics.

Responsibilities

This section allows you to put the skills into context.? To continue the example from above, designing PowerPoint for an internal audience of fellow employees, board members, and other stakeholders is different than designing PowerPoint for externally facing webinars, talks, or events at large conferences, or as part of a video that potentially millions will view.? Be sure the candidate understands in what context they will be asked to use the skill.

Often, the validation process is either missing or incomplete.? Beyond telling a candidate where to and how to apply, you should also explain how you plan to evaluate the claims candidates make in their resumes.? What happens after they apply?? Will the evaluation be a verbal Q&A?? Will it be based on a portfolio of past work?? Will it be an objective test?? How many stages are there in the validation process?? How long will the process take???

If candidates know the process in advance, they can agree to it by applying, or choose not to apply.? However, if a candidate applies, and then only afterward becomes aware that they might be asked to spend an hour taking a test, or asked to do unpaid pre-work, they will become frustrated, may withdraw their candidacy, and possibly report their bad experience to other potential candidates.

Here is an example from a job description that explains the full process (although there is room for improvement: evaluations are subjective and no timeline is provided).

“Applicants meeting the minimum requirements will be phone screened about their relevant experiences to assess their knowledge of the subject matter and required skills.? Those selected to move forward will be asked to present examples of past work which will be evaluated by the hiring manager for thoroughness, clarity, and innovation.? Those whose samples are exemplary will move to a two-part interview with the team and management using a common rubric to rank and evaluate the final candidates before an offer is made.”

If you would like to speak with someone about improving your company’s validation process, please contact [email protected].