A Better Way To Assess Exceptional Vs. Average Performance In An Interview

I want to give you a blueprint for what I believe based on my experience and research is the best way to structure your interview framework to produce a more optimal outcome to identify a higher-performing employee.

If you’ve read me in the past, you know I like to get as specific as possible so you have tangible and actionable to takeaways.

I also need to be a little blunt with why I felt compelled to right this article, as I believe a lot of you will share my same concerns. One of the great things about the internet is it allows you to find and get access to so much information to help you find better ways to do your job. The flip side though, is if you did a search on Google for “Best interview questions” you get 460,000+ results. Not that I have looked at each of the 460,000+ results, but like you, I have seen my fair share of articles over the years from people that do more harm than good on this subject.

Don’t get me wrong. I am not the smartest person in the room. I don’t have all the answers. I do though think that it’s time for me to share and put a stake in the ground around what I have learned about different interviewing models, questions, and better outcomes. Sorry folks, I just can stand seeing another article about “The 10 best interview questions in 2017” or hearing from people like that dude in “Something about Mary” talk about seven-minute abs. Kill me now.

What I am going to share with you in this article is based off my own trial and error. I even wanted to get some opinions from two well-respected people who have personal passion and detailed experience, Nathan Mondragon and Jim Durbin, who have specialized in optimal interview models and approaches in our industry for decades.

Nathan has been doing this for years and has lead large teams of I/O psychologists for some respected companies. He will also probably tell you that while there are 460,000+ articles online from Google on the subject, most of them are very dangerous in what they suggest and are nothing more than click bait. Nathan will also give you a very interesting perspective of what is the future of interviewing and what companies are doing today using machine learning algorithms for interviewing.

Historical Choice of Interview Question Examples

Open and closed questions: Tell me what you think about x = open. Do you prefer x vs y? = closed.

Gotcha questions: What are your strengths?

Standard or common questions: Why are you interested in this job?

Riddles or brain teaser questions: Why is a manhole cover round?

Functional questions: What was the most challenging project you worked on, and why?

Technical questions: How would you write a function that can reverse a linked list?

Cultural and team fit questions: What company culture do you thrive in, or what company values are you drawn to?

Competency-based questions: Tell me about a time when you had to deal with a difficult customer. What was the issue, and tell me how you specifically resolved the situation.

The final one is behavioral-based questions. I am going to get to this one in a little more depth below.

Ok, let me cut to the chase. Pretty much all of these questions are crap at identifying if a candidate will be successful at the job your hiring for in your company.

The Elephant in the Room

So, if we agree they are crap at assessing performance, why do we still ask them?

Simple. Because they are easy. It takes no formal training or real skill to ask some asinine question that you read from some previous online best practice top 10 list article.

Interviewing people correctly is hard. It takes time to learn. It takes time to put a structured interviewing framework together … and probably the biggest elephant of them all, in a room full of rogue bull elephants, is trying to convince dozens, or hundreds, or thousands of hiring managers to do it differently than they are used to.

The Real Problem We Are Trying to Solve

In the simplest terms, the problem I believe we as an industry are trying to solve for, is how can we (business and HR as well) do a better job interviewing and assessing a candidate who end up being a high performer in an organization.

Let me give you the TA leader spin on this as well. How can I reduce the amount of time my team spends (more that 50 percent in a lot of cases) having to focus on backfill requisitions so they can spend more time doing added-value work?

Yes, there are post-hire elements out of the control of recruiting that come into play here, but before you shut down, bear with me.

If we agree with this mission statement above, then we must also dig into why current methodologies we use are no better at predicting success, or it’s akin to placing a bet on one number at the roulette wheel in a casino.

Let me give you a real-life, practical example we see often when recruiting. If you do not agree with this example or logic, then you should not read on.

The Constant in This Story

Have we seen candidates (or even worked with people) who excel in a large organization but then fall short when they move to a smaller startup?

Have we observed people who seem to thrive under one manager, and struggle under another?

So, what is the constant here? The culture? The manager? The industry they work in? No, the constant is the candidate or the employee. People move jobs, they get new managers. Why do some people seem to more effectively transition more successfully from one scenario to the next, while other people struggle?

When I have analyzed this in the past I found that the constant on why people don’t succeed in role and in companies comes back to their “soft skills.” When I talk about soft skills, I am referring to the competencies and behaviors that a person does or does not demonstrate. Let me give you some examples for practical context of what is a “soft skill” competency:

• Dealing with ambiguity
• Building collaborative relationships
• Results orientation
• Analytical thinking

Nathan will tell you that in the world today there are about 30 core competencies. Yes, some companies might call it realizing results, where another might call it results orientation, and another calls it results driven. The bottom line is they are basically saying the same thing, just they are putting their own twist on that core competency. That’s what companies like to do, because they like to be different and special :-). 

Now I would think if you have been in recruiting for a little while you have come across the concept of competency-based interviewing questions. While foundationally competency based interviewing is clearly better at assessing performance compared to “why is that man hole cover round, buddy?” they still fall short, IMHO. If people clearly don’t define and assess the difference between average vs. exceptional performance, then you’re not getting to the constant aspect of the problem.

A Better Interview Framework

I alluded to behavioral questions above, so now let me give you details as to a way that you may not have not thought of applying behavioral differentiators against the key competencies of a job to produce a better outcome. The quickest way to explain this is by showing you a competency referenced above to build the foundation of the framework from design to application.

Step 1: (Current state — the world today for most of us)

In your organization, we will say you have a need to hire sales-people. Normally when the SVP of sales wants to hire a new salesperson, they will send recruiting a job description. Most job descriptions, like this one I found online, will contain something like this in the skills/requirements section.

I intentionally highlighted the desired skill (competency) of results orientation in red for this exercise. The issues with job descriptions like this and the downstream interview questions is that they really don’t define in any meaningful way what average vs. exceptional performance looks like. Oh, that’s right, we leave that up to the hiring manager, who says, “keep sending me people, as I will know what great looks like when I see it.” Yeah, we know that approach works, right?

Let’s use the listed point above of dedication to follow through to ensure results.

• Do I have to be dedicated to follow through?
• What results am I specifically ensuring?
• Is the urrent state that most of the salespeople ensuring results are doing this on average 60 percent of the time, so if I can do 70 percent, would I be exceptional?

Step 2: (defining future state)

In my experience, the first step in this journey is sitting down with the hiring manager to:

• Identify the key competencies “soft skills” that they want to hire for in this role, and why?
• Most importantly, define the practical behavior examples of a person they define as exceptional, who consistently demonstrates proficiency in that competency vs. those that are just average.
• Prioritization of said competencies; otherwise you could be on the hunt for that mythical purple unicorn on rollerblades candidate who does not exist.

If you are going to take on this initiative at a larger scale, with multiple hiring managers in a business unit or across the whole enterprise, I will get to that in a bit. Let’s build this out for one hiring manger as the foundational example first.

In this case when you sit down with the hiring manager, they come back with three critical “soft skills” (competencies) they believe are critical for the role: results orientation, customer focus, and effective communication.

The first thing you want to do using the results orientation competency in this example is ask the hiring manager to explain the difference in their experience of a salesperson in this role who consistently demonstrates the right behaviors that make them exceptional vs. salespeople who are average.

I want to pause on the word “consistently” for a moment, as it’s key.

What I, Nathan, and talent management functions have found across the majority of people who demonstrate exceptional behaviors vs. average behaviors in a competency, is that the exceptional behavior shows up regularly. They do it consistently. An average person might demonstrate that behavior once or twice a year, but that does not make them exceptional and proficient in that competency and associated behaviors.

This is where I have found that traditional competency-based questions fall short. They ask and phrase the questions something like this: “Tell me about a time when you were dealing with a difficult customer.”

See the issue here? Tell me about a time. One time. We need to get at the demonstrated behavior consistently showing up and we are not going to get there by asking competency based question that directs a one-off example.

We are now in a great conversation with a hiring manager and we are starting to flesh out specific examples of what the behaviors look like when defining results orientation. I have found this visual below helps start define what average vs. exceptional behaviors look like for the competency of results orientation for this sales role. Take the time to look at the difference in the language between average vs. exceptional highlighted in red.

I don’t want to turn this article into a competency-development framework discussion, as depending on your company size, you are clearly going to need to use HR and a talent management function, or you could be using an existing performance management framework with an existing competency model.

For those of you who are not familiar with competency models, here are some basic principles. I referenced above that you could be on a path to building this interview framework out for dozens of hiring managers, or even across the enterprise. Keep the following in mind if you are on that journey:

• The successful behaviors that get demonstrated by a person in a role clearly change as the experience, scope, and seniority of the role change. Practical example: The behaviors you might need an associate salesperson to demonstrate for the results-orientation competency are going to be different than exceptional VP of sales behaviors. While you’re looking to identify those exceptional associate salespeople to demonstrate the “consistently exceeds sales plan,” for the VP of sales clearly you’re looking for “consistently manages and motivates the team to consistently exceed their sales plans.” Clearly how we are going to structure a behavioral-based interview question is going to shift as well.
• The other thing that I have done in the past is not just take a few hiring managers at their word for what they believe is exceptional behavior. Truth be told, if you don’t demonstrate some critical thinking here yourself, it could be the blind leading the blind. Go and interview top-performing salespeople in your organization and identify the common theme behaviors that they demonstrate. But don’t stop there. Go interview lower-performing salespeople in your organization and do the same. What you will have is a better-established base of differentiation, and remember at its core that is what we are trying to do — identify great vs good.

[self-editorial note for some of you readers]Yes, the world and companies are not made up of all exceptional performers. Yes, we need plenty of Steady Eddie workers. But, I would be shocked if reading this, even one of you thought that our roles, our mission, should not be about helping companies do a better job of more effectively interviewing and assessing the difference between good vs. great.

Step 3: (Landing this in the field)

Now that we have defined the framework of what good vs. great can look like, we now must turn these behaviors into viable questions. Those of you experienced in question design know that the phraseology of a question (input) can clearly influence the answer (output). This does take some skill and for people like myself who hate Twitter for its 140-character limit; it takes me a little more work. Hey, I’m a long-form content and details guy. 

Here is a template with some question examples that I have created for recruiters and hiring managers in the past that can give you some ideas of your own.

Let me break down the template above:

1. The top cell is the definition of the competency and the behaviors you are trying to assess.
2. The ask section is just that.
3. Re: the Listen For section … as an interviewer, these are the key evidenced by behaviors items you have defined with the hiring manger (or HR competency model) that you want to listen/hear the candidate demonstrate.
4. Re: the assess section. After listening to the answer, with potentially additional probing questions, the interviewer indicates did they meet, consistently exceed, or did not demonstrate proficiency:

• Did not demonstrate = Gave no examples at all that touched on any of the key evidenced by behaviors.
• Meets = Gave an example, but could not show consistent proficiency in all the behaviors.
• Consistently exceeds = Gave multiple examples that demonstrated all the evidenced by behaviors.

5. Comments section. This is where an interviewer could provide optional additional notes, comments, and recommendations. Example: The interviewer might recommend that the next interviewer or hiring manager dig into a behavior in more depth given an answer from the candidate.

A Talent Advisor Initial Hiring Manager Meeting Template

Now we have covered the foundation, I am including a kick off call (intake, strategy meeting) template where you can connect all the dots back to a behavioral-based, structured interview framework approach.

You can download it here.

One comment I have on it:

Rob has done a perfect job in blending the science and application with his competency framework and interview evaluation approach. Too frequently people get bogged down in the “decimal dust” or “splitting hairs” over multiple competency levels and too many rating levels. Trained evaluators are good at making decisions across three levels (bad, average, great), but lose their accuracy when more levels at thrown at them. — Nathan Mondragon, chief IO psychologist, HireVue

Some Additional Thoughts and Bonus Points

I want to provide a couple of noteworthy items related to the above, and also some connected concepts when you think about building a structured interview framework in your organization. I won’t go into huge depth here — otherwise this article will start turning into a novel — but it is worth calling out as you think about your own journey.

• Structed interviewing approach. If you are all familiar with the term structured interviewing, then each candidate interviewed must be asked the exact same question. Use the Talent Advisor Template I linked to above to help assign competencies/behaviors across the business interview loop. We do this so we can effectively assess candidates against the key job requirements, not just one candidate vs. the other. If we don’t do this. then we are back to comparing “apples vs. bananas,” and truth is, this is one of the major flaws with interviewing models today. As to my point stated below, you can have structure and flexibility happily co-exist in an interviewing framework.
• Flexibility in delivery but not at the expense of quality output. I am not suggesting that each interview is 100 percent dedicated to asking structured behavioral questions. You can assign these across different people on the interview loop and they can be part of the overall interview process. The recruiter/sourcer can take the most critical two core competencies/behaviors as part of their pre-screen. Even if the structured behavioral questions take up 15-30 min of the interview, you still have time to ask, “how many ping-pong balls can you fit in a 747?” if you like those questions. Just make sure that someone is consistently asking and assessing the core behaviors against the success criteria of the role, that’s all.
• People are too busy or just lazy enough. One thing I noticed in the past ,and you have felt it too, is trying to get hiring managers to provide comprehensive interview notes is like pulling teeth. One of the things I wanted to solve for with the above interview template (Results Orientation Competency) is trying to mitigate the burden of writing copious notes as part of an interviewer’s feedback responsibilities. Also, leaving interviewers to come up with their own behavioral-based interview questions was also very dangerous, particularly when there was no structure in place. When I tested this theory with business executives, they agreed that we needed to 1) Provide a simple to follow consistent guide where the defined behaviors had already been validated, and 2) Have a framework that requires the interviewer to focus more time on listening and then simple click the appropriate check-box based off the candidate’s answers.

The Halo effect. Never, ever, ever, have people on the business interview loop provide public hire/no hire decisions to the rest of the interview group.

The hiring manager, working with either the recruiter/HR, should be the ones collecting the feedback to make the ultimate proceed to the next interview decision or not. If you are not familiar with the “halo effect” as it relates to interview feedback psychology, do a little research as to its negative impact on interviewing if you are not familiar.

• A scoring scale from 1-5 is foolish. If you create a scoring scale for interviewers to rank candidates, what ultimately ends up happening, is it’s challenging to clearly define and then get everyone to agree on the definition of what equals a 1 vs. 2 vs. 3 vs. 4 vs. 5 score. One of my favorite sayings from the wise Gerry Crispin: If you try and set a standard or measurement system, be prepared for the majority of time to be spent on debating the definition.

Not to get all HR on you here, but number ranking systems that are subjective with no proven correlation to job requirements criteria are a very slippery disparate impact slope. The closer you can get people to think in terms that people either:

1. Did not demonstrate the behavior
2. Demonstrated the behavior

… The better off you’re going to be.

The more linear you can get, the more you also start to remove the historical crap that I have personally had to wade through my entire career where you see business interviewers make decisions recommendations like: “hire with reservations.”

That’s B.S. and a cop out. Step up and make the judgement call. They either meet or do not meet the key evidenced by behavioral success criteria of the role.

If it’s worth your time, it’s worth your attention.

• Sourcing for exceptional performers. As I said, I don’t want to turn this into a novel, but I can give you something to ponder if your brain is thinking how this approach can be applied to proactively trying to source exceptional performers. Let’s use our sales example again. Ever thought about adding the following terms to your search string? (“exceeded plan” OR “surpassed plan” OR “beat plan” OR “exceeded target” OR “exceeded goal” OR “presidents club” OR “Top 1%” OR “Top 5%” OR “Top 10%”). I think you get the gist of where I am going with this. I have personally taught my sourcing teams in the past to research competitor companies to find out the specific names of company awards that are given their exceptional employees. Heck, LinkedIn makes it ever easier with a specific section that candidates use on their profiles, and we know people like to show off. 

I don’t know about you, but I would prefer to add a string like the above to my traditional keywords just to potentially bring back the top-performing profiles first.

Some Other Thoughts Beyond Mine

Here’s what Jim Durbin, digital marketer and headhunter with Brandstorming, says:

The modern office moves very fast in comparison to even a decade ago. The challenge we face in any interview process is a lack of time and attention. Human beings tend to believe that what they’re focused is on is what’s most important. (This is why meetings are derailed with a ten-minute discussion on one phrase in a 30-page presentation). When you leave a meeting at 1:59 and have an interview at 2:00, you will find it very difficult to immediately switch your attention to the interview. This is known as “attentional focus.” You have to clear out and close what you were working on, or your very human brain will lack the ability to connect with that candidate. One things human do very well is read and react to non-verbal cues. A disinterested manager is going to get less useful answers than a manager who is focused on the interview.

Look at it from the standpoint of the jobseeker. “Let me take a look at your resume,” or “let’s pull out this interview script and see what it says” suggests that this interview is not an important part of your day. That lack of focus is equivalent to a candidate responding to your question about the company with “from what little I know …”

You would rightly conclude the candidate was either unprepared or lacking in confidence. What are they concluding about you?

It’s not enough to be trained well and follow the process. You have to respect the candidate enough to come to the interview prepared and ready to engage. This means scheduling a few minutes prior to the interview to get your thoughts together.”

Nathan Mondragon, chief I/O psychologist, HireVue, says:

If you follow the advice Rob provided in this article, you should obtain higher quality results, and not to mention, be more legally defensible and unbiased in your interviewing methods. For decades, structured interviews have been a valuable method for eliciting high fidelity, rich job-relevant interactions. They balance the ability to predict important job and organizational outcomes with a more positive, engaging applicant experience. However, scalability of these interview programs is hard and predictive accuracy decreases as more interviewers are used (there are more bad evaluators than good ones out there).

Even if you don’t feel ready to build a better interviewing framework right now, hopefully you can pull a few nuggets from this article and start that dialogue with HR and the business.

Need some Intelligent Advice on how to? Come visit me at McINTOSH & Co.