Measuring Candidate Experience

• Understanding the talent acquisition journey from the perspective of candidates
• Measuring the influence of candidate touch points
• Creating a measurement framework to produce insight and continuous improvement in talent acquisition and attraction

Imagine you are new to a neighborhood and someone invited you and your spouse to their home to have dinner. You were excited to go. Now imagine that you showed up and nothing was prepared. They seemed rushed and decided to throw together some leftovers on the fly. They didn’t ask you your food preferences, and they served something you couldn’t eat. Graciously you smiled and ate the other items on your plate. They spilled wine on you. You didn’t seem to find anything in common with them, and they scrutinized the areas of difference in great detail. It didn't seem they understood you, and you felt judged. They got distracted and ignored you for part of the evening. Strangers came and went throughout the evening. Some were odd. As you and your spouse are leaving you think you hear them whispering behind your back. As you and your spouse walk home you are frustrated, exhausted and a little afraid about this neighborhood.

What I just described is kind of like the experience of many candidates, maybe most candidates.

Perhaps, evaluating talent acquisition from the perspective of the candidates is enough for you in its own right. If not, there are some other reasons that candidate experience is important.

First, recruiting touches many people. These people walk away with an image of the company. They hold on to this image and they share it with their friends and family (and increasingly they share it with the world).

Second, recruiting is the first experience the people you are going to hire have with your company. It says something about your company, and this frames your future journey together.

Third, recruiters, hiring managers, hiring committees, candidates and the collective company share some common objectives and have some divergent objectives. If we don’t measure things then we have a hard time holding everyone accountable to do the right things for the company. Maybe the incentive for the recruiter is to turn hires out quick and move on to the next – produce hires without regard to quality or care for people. We need the recruiter to not just produce hires, but also produce the best quality hires possible and do this without leaving a trail of dead bodies. Same goes for managers and hiring committees. We all have blind spots. We all have a lot to do and left unchecked we get caught up in what we are trying to accomplish. Feedback instruments allows us to see how our actions impact others and our team and company objectives more clearly.

Finally, candidate surveys should not just be a smiley sheet survey or passive aggressive way to measure recruiting team performance. Candidate surveys have much more potential.

-      You can measure process consistency,

-      you can measure bias,

-      you can use surveys as a counterbalance against potentially inhumane time, cost and quality measures,

-      you can measure the other things that people do in the interview process that degrade the opinion of candidates that have nothing to do with recruiters (otherwise unseen or heard),

-      you can learn more about candidate motivations and situational reaction so you can target better in the future,

-      you can learn how to improve your recruiting process, teams and your company, and

-      you can infer information about the companies that you compete with.

Broader and longer journey

Occasionally we talk about the “employee journey” or “employee life cycle” in human resources.

Employee Journey is the long journey people make from first contact with the company through the recruiting process, new hire orientation, on-boarding, first 30, 90, 180 days, on to the first anniversary, extended anniversaries, until the point of exit. The length of this journey varies by individual, job and company characteristics. If someone is able to make a career at this company the journey may be much longer than someone who works for your organization as a job until they figure their personal journey out.

We will just be talking about the very first leg of this journey, which shapes the rest of the journey.

There are variation concepts but sometimes Employee Lifecycle ends up referring to the ways human resource management connects to the collective employee experience through our annual work in: headcount planning, recruiting, performance mgmt., compensation planning, succession planning, etc. Sometimes these are illustrated as a wheel or cycle. Indeed it is a sort of cycle but I am not sure how useful this is right now. We will talk about this another time and day. I will not discuss this in this article.

The broader journey and actions of human resources matter, however, we will discuss the mathematical reasons why everything we do in human resource management should thought about as a single system later. I will not discuss this really in this article.

We also could get into the mathematics of the recruiting funnel and the relationship to Hiring Throughput, Time to Hire, Cost of Hire and Quality of Hire. Of course, these all effect relate to each other and are all important for different reasons. As any consultant will say, “Done Right, Cheap, or Fast: choose any two." You can’t have them all. I will not discuss these important counterbalancing measures in this article.

In my professional experience, I have found that focusing first on collective experience of human beings has never led me wrong, has always made a profound contribution to whatever the company was trying to accomplish, creates compelling stories, and somehow is less complicated than other methods. So I start there today.

The scope of the current article is establishing a measurement driven learning system from the candidate perspective from the very beginning of the employee journey, which we will use to produce learning that will help us build and refine a model of "Attraction".

For a brief catchup, Attraction is one of three "A"s (Attraction, Activation and Attrition) that form the primary people needs in organizations - as defined in my philosophy of Lean People Analytics. My goal is to systematically build out the complete scientific and mathematical structure for the whole, but I am starting in the focus area of Talent Acquisition Analytics in anticipation of a live online workshop starting this week (signup quickly and tell others). If you miss it, let me know and we will get you in the next one or catch you up with recordings.

Learning from a traditional marketing & sales funnel

There is value in borrowing from the fields of Sales and Marketing. This way we don’t have to start from scratch. People have been working on these concepts for a long time, and they are stacking analytical methods on top of an already solid foundation. there are college degrees, classes and many books on these topics. I want to borrow that foundation until we have a reason to replace or modify. We will, but let’s not reinvent the wheel just yet. We are going to look for what is useful first.

Here is how we think about the customer journey - from not knowing anything about us at all to a sale - from the standpoint of marketing.

The funnel illustrated above demonstrates that all customers move through a similar set of stages from first awareness to purchase. The example also illustrates the funnel is larger at the top and smaller at the bottom – this indicates many people don’t make it through the funnel. The product doesn’t meet their needs, doesn’t compete well with the other options they are aware of or the company has failed at some stage to move the potential customer forward. 

You could add to this Marketing Funnel a Sales Funnel, which would expand in more detail the company touch points (often systematic) that takes the customer through consideration until purchase. This would be more naturally observable in enterprise sales (or otherwise more complex sales) than consumer products like toothpaste, for example. We kind of just expect people to stumble their way through this journey to our toothpaste, but if we are selling a hundred-thousand-dollar device then there is more extensive handholding and systematic treatment of the potential customer. Talent Acquisition is more like the complex or enterprise sales process. Considering the full lifespan of an employee with the organization we are selling hundred thousand dollar (if not million dollar) products to people, and we are simultaneously buying them. Pardon the metaphor.

In any case, Talent Acquisition has a similar funneling phenomenon as a sales process. We create awareness of opportunities through passive channels (PR, advertising, job boards, LinkedIn) and active sources like sourcer and recruiter outreach, which is more or less the sales arm of human resources.

Divergence from a traditional marketing & sales funnel

An important difference to observe from sales and marketing is that there are actually two funnels occurring at the same time. The world of potential candidates funnels down into a possible relationship with the company (their choice to proceed), while at the same time the company is narrowing the range of potential candidates to select their perception of best of the available choices. Note that increasing the candidate pool is inherently useful to produce more high-quality options, but if it is done indiscriminately then it just creates more candidates to select from, which could result in lower quality hires if the lower quality candidates block the higher quality candidates.

Unlike a sales funnel, in a recruiting funnel the problem is passed back and forth between two funnels.

In this article I will not discuss much the mechanical process implications dual funnels (Cost of Hire, Quality of Hire, and Time to Hire), however there is an important connection and what you are doing in Talent Acquisition Analytics is therefore actually richer and MORE COMPLEX than what is occurring in a typical Marketing Analytics function, AND you are doing more with less resources. 

The response from sophisticated Talent Acquisition functions is to divide responsibility through different roles: Sourcers, Recruiters and Hiring Committees to deal with the complexities and different needs. Having a specific role for sourcing, acknowledges the need for Outbound Sales to acquire the attention of people with specific capability and quality features, which then produce a segment in the hiring funnel that will have a much higher and cost-efficient throughput. Recruiters job is as sort of mediator and human contact to walk everyone through this journey together, and Hiring Committees job is to be a stop gap for quality. All of this can be measured.

The survey-based measurement system I propose in this article will provide some important insights about how well Talent Acquisition is doing on both sides of this funnel and will help to direct attention to places where more complex analytics will be more or less useful by inferring phase and segment pains.

I recognize that survey data is subjective, and the winds of the human mind and soul are illusive, however we are using this data in aggregate, which has useful mathematical properties, including ability to predict objective outcomes we care about. See Fueling People Analytics with Feelings. Don’t worry about the individual detail of it, or the wrong or right of the individual responses. Even if you wanted to see the individual detail you shouldn’t see that detail because then nobody would give it to you. We are looking in the aggregate and we are looking for patterns.

What I care about from survey data is:

1.     the aggregate meaning of the averages and distribution illustrated by some method of classification/segmentation,

2.    how the averages and distribution move over time,

3.    what other outcomes these data mathematically associate to,

4.    and the effectiveness of the techniques we apply to move the averages by segment.

Organizing analysis with a Talent Acquisition Journey Map

A Customer Journey Map is a visual document that charts the customer experiences as they progress through the stages of a companies marketing funnel to purchase. The Journey Map brings together key interactions that the customer has with the company, but it also the customers feelings, motivations and questions at major touchpoints. The customer journey map for service design was first introduced through the Acela high-speed rail project of IDEO (1999). It has subsequently become one of the most widely used tools for service design and have been utilized as a tool for visualizing, analyzing, communicating and improving intangible services. Here is a paper on Journey Maps.

The goal of the Journey Map is to define the path that customers take for analysis to understand how prospects and customers experience the various channels and touchpoints and how the company is perceived relative to how the company would like the experience to be. By understanding the customer point of view through the journey it is possible to design an optimal experience that meets the expectations of major customer groups to produce advantages.

Journey Maps can be high level and simple or detailed and complex. Here is an example Customer Journey Map from Vivian Jacob shared on a website called "Behance".

You can also apply a Journey Map to the candidate journey. Below is a measurement framework I created that connects the major touchpoints of recruiting with candidate pools and the journey of candidates from awareness to offer accept. This a technical map and doesn't convey a key segment experience as the example above does. I encourage you to use it to frame your effort to obtain information to create a good Candidate Journey Map for your company, which will be creative and unique. While you can find plenty of great customer journey maps on the internet, I have yet to find a clear and concise illustration of how you acquire the information you use to continuously evaluate your journey map. What I have below is an attempt to illustrate the technical architecture to do this - applied generically. I encourage you to create your own based on your own unique recruiting process. I didn’t fill in the expected emotional responses or other elements because I don't have that information for you, but if you diligently collect it you will have it. I indicate the key measurement devices and metrics I would implement at each stage, which would allow you to accurately measure emotional reactions for continuous feedback for improvement.

Creating your own Journey Maps

I might do a future article on this topic, but for purposes of getting you started, the steps are:

1.    Pick a Key Persona Segment x Key Job Family Segment combination (you should do one for each major segment combination unless they are not coming up different)

2.    Determine the stages that apply. Consider the goals of the candidate and how they align or vary from the goals of the company. You may indicate both.

3.    Define steps.

4.    Define key company touchpoints with candidates.

5.    Identify the key information needs and questions a typical candidate experiences at each stage. You should also consider the information needs and key questions you or the hiring committee have at each stage. You will need to ask various parties for this information.

6.    Define Metrics for each stage.

7.    Identify the pain points. Examples: “In the consideration stage, candidates misunderstand the companies pay and benefit structure.” “In the preference stage, candidates can’t differentiate the job opportunity, except by level of pay.”

8.    Identify who is accountable for each stage.

9.    Analyze to uncover problems and opportunities.

10.  Periodically validate.

Hiring Manager / Hiring Committee

I suggest that you create a separate Journey Map to represent the journey of the hiring manager or hiring committee from initiating the services of the Talent Acquisition team through fulfillment with a candidate and evaluate this against realistic conditions.

If the reality is that you have to take several passes to produce a viable candidate, then this iteration should be reflected in the Journey Map also.

If you are creative and you have a large enough space to work with you might be able to get all experiences in one joint Journey Map. 

How to use the Candidate Journey Map in analytics

The Journey Map is not an analytical tool per say. The Journey Map is a design and communication device. This said, working with the Talent Acquisition and Hiring Managers to visualize each step they and candidates go through will help to clarify the main stages, the expectations each stakeholder has about the experience and the touch points that can influence the experience in those stages. All of this is useful to help you design measurements and know the reference points you have for experience, time, quality, cost and throughput metrics.

The map is not the end goal, but having the map helps you frame the analysis and is a great storytelling tool so you will find it well worth the time you invest in it. You might find valuable learning and change while creating the journey map, before you have even applied any advanced analytics.

Using surveys to infer insights across the journey

Candidate Survey Question Database

In the Candidate Survey Questions Database section below are example candidate survey questions that confirm to a Likert Agreement Scale by category.

Clearly, I do not suggest you should ask all of these question examples in the questions database below on a single survey. You should be selective, and you should spread the survey workload out. See “Designing Candidate Surveys” below this section.

Lightning Index

I suggest the “lightning index” as a base before and after onsite interviews. The lightening index is about 10 questions that provide inferences about the process, the company and the recruiter (while not explicitly naming the recruiter).

-      I know everything I need to know about the opportunity (at this stage).

-      I know everything I need to know about the hiring process (at this stage).

-      The opportunity that is described to me is unique.

-      The opportunity that is described to me is compelling.

-      I have had the opportunity to describe what is unique about me.

-      I have been provided sufficient information to prepare for the next step.

-      I wish more companies had a hiring process like (insert company).

-      If I was going to go start my own company, my recruiter at (insert company) is the first recruiter I would hire.

-      If I were given a fair offer today, I would take it (hypothetically speaking).

-      I would recommend other people I know apply at (insert company).

Creating Insane Recruiters

These are questions that explicitly name the recruiter. These are examples:

-      The recruiter spent just the right amount of time with me.

-      The recruiter described the opportunity carefully to me.

-      The recruiter was surprisingly knowledgeable about the space in which I work.

-      The recruiter expressed interest and curiosity in learning more about the work I do.

-      The recruiter took time to listen carefully to me.

-      The recruiter provided helpful context about the company, division and team.

-      The recruiter was able to answer all of my questions (or followed up to get me answers).

-      The recruiter was clear about the process and next steps.

-      The recruiter followed up with me in a timely way.

-      The recruiter gave me the information I needed to get to the right location.

-      The recruiter gave me the information I needed to prepare for the interview.

Creating Great Interview Teams

These are questions that measure the experience of the onsite interview. These are examples:

-      The interviewer(s) showed up on time.

-      The interviewer(s) made me feel welcome and as comfortable as I could be at an interview.

-      The interviewer(s) were well prepared to speak with me.

-      The interviewer(s) were knowledgeable about the line of work I do.

-      The interviewer(s) were interested and curious about me.

-      The interviewer(s) explained and applied an interview method designed to reduce bias.

-      The interviewers were diverse.

-      The interviewer(s) have clearly defined what the job is.

-      The interviewer(s) have expressed a unique selling point for the job.

-      The interviewer(s) have realistic expectations about the job.

-      The interviewer(s) have realistic expectations about candidates.

-      The interviewers all asked me different questions.

-      The interviewer(s) seem to know what they are doing.

-      The interviewer(s) seem to be "up with latest", moving forward rather than backward.

“Process This” Feedback

These are questions that measure the experience of the process. These are examples:

-      The hiring process at (Insert Company) is much better than my experience with other companies.

-      The hiring process reflects well the (Insert Company) stated mission, values and brand.

-      This hiring process has been challenging but fair.

-      The hiring process at (Insert Company) seems well designed to produce high quality hires.

-      The hiring process at (Insert Company) seems well designed to reduce individual bias.

-      The hiring process at (Insert Company) seems well designed to work as quickly as I can imagine it should.

-      The length of time between when I started the process and when I spoke to someone on the phone was reasonable.

-      The length of time between my phone screen and onsite interview was reasonable.

-      The length of time between my onsite interview and when I received my offer was reasonable.

-      The length of time between when I started the process and when I finally received my offer was reasonable.

“Image That” Website

These are questions that measure the experience with the career website. (obviously you would only use these in situations where they apply) Examples:

-      The company career web area is substantively better than that of other companies.

-      The company career web area expresses what is unique about this company.

-      The company career web area is a place where I would actually go again.

-      The company career web area provides useful information.

-      The company career web site provides a convenient channel to get information.

Open-Ended Questions

-      How can we improve our recruiting/hiring process?

-      What aspects of the opportunity are most compelling to you?

-      What aspects of the opportunity are a concern to you?

-      When you think of our company brand, what words comes to mind? (List as many as you can)

-      What concerns, if any, do you have about (Insert Company) and its future prospects?

Employment Brand Questions

In a previous article we established a framework for measuring employment brand. Rather than go back over this, I recommend you review that article here: Measuring Employment Brand & Attraction.

It is important to run some brand survey samples among key job segments before inviting people into a recruiting funnel to help you learn about all potential candidate features and motivations, not just the ones that agree to come with you on the journey. If you don’t take any brand samples prior to inviting people into a process, then you won’t get data from those who decline these recruitment efforts outright. If your goal is to learn how to improve your odds of getting candidates to enter the process then you will need some analysis that is independent from the process itself so you can differentiate the response profiles of those who enter from those who don’t. Having both provides the data you need to learn how to more optimally target and message to produce more candidates. Again, this is just a quick summary: see my article on Brand & Attraction, hyperlinked above.

With the caveats provided, brand measurements also appear useful through the Candidate Journey. If you include a brand item in your periodic candidate surveys or run random sample checks you can measure “Lift” on that item. Lift is the difference in some attitude or opinion measure before and after a touchpoint experience. For example, you may measure brand concepts among a sample of people prior to inviting them to discuss an opportunity, just prior to onsite interview and then again after. This will show you if these encounters have any effect.

A positive lift indicates you are improving the perception the candidate has of the company brand as they go forward. A negative lift indicates you are degrading the candidate’s perception of the company brand.

Here are examples of brand questions:

-      I have a clear understanding of (Insert Company)'s brand identity and products.

-      I have a clear understanding of what differentiates (Insert Company) products from competitors.

-      (Insert Company) products and services are generally as good as, or better than, competitors.

-      (Insert Company) is in a position to really succeed over the next three years.

-      (Insert Company) has meaningful mission, values & products.

-      (Insert Company) is highly regarded by the general public.

-      (Insert Company) compares favorably with competitors as an attractive place to work.

-      (Insert Company) has a reputation for treating people equally regardless of ethnicity, gender, age, sexual orientation, religion, or disability.

-      I would recommend other people I know apply at (Insert Company).

-      I have confidence in the leaders at (Insert Company).

Designing Candidate Surveys

Survey design is an actual science. Some may say it is a science and an art – I totally disagree. It is a professional skill-set – if that is what they mean by art, then o.k. I agree but to me the word art means something different. Almost everyone thinks they know enough of what they need to know to design a good survey. Only in rare exception is that actually true. The more you learn the more you see there is a lot to learn about the human mind and surveys. It is a professional craft. If this is not what you have been trained to do, it is worth a few hours of consulting support and/or review from someone who is professionally trained in survey design and administration.

Moving forward anyway…

First, I do not suggest you ask all of these question examples in the questions database above on a single survey. You should be selective, and you should spread the survey workload out through samples.

I provide a range of questions to choose from because I know the hardest part is thinking of the possibilities and designing them within a consistent scale when under time pressure.

Choosing what questions you should use among the many possibilities is a function of your theory of what matters. It also reflects specific topics you know are relevant for the company to work on right now and others you know are not. How much insight the survey produces for the company is a result of how well you did this. If you design the survey that is on you, not anyone else. Don’t be too hard on yourself, apply iterative design review to make the survey produce better insight over time.

Survey Design Strategies:

-      One strategy is to use the “Lightning Index” on all candidates before onsite and after onsite and just do periodic tests of other questions.

-      A second strategy is to bring questions in or out of your survey focus as needed for specific research or A/B test objectives.

-      A third strategy is to use a series of brief surveys after each touchpoint in the candidate experience, repeating some questions through the process to observe “lift” and others that are just specific to each touchpoint.

-      A fourth strategy is to keep your survey check-ins in the process brief, but provide a more extensive survey at the end of the process – before an offer or rejection.

-      A fifth strategy is to keep your survey check-ins in the process brief, ask a few more questions prior to accept or reject and then provide a very extensive survey during onboarding phase – to those who have accepted.

The strategies above are not necessarily exclusive. You can mix and match.

Windows of opportunity. At the hire and onboarding stages, people are engaged in their new company and energized to provide feedback. You want to use this time to learn as much as you can because you have enthusiastic subjects. Also to honor enthusiastic subjects. These people have just decided to invest their foreseeable future in this company. Asking them how to do things better honors their commitment and enthusiasm. Not caring enough to ask what they think says just the opposite: it’s say’s thanks, we got one up on you, and we don’t care anymore.

Scale. I have designed all questions on a Likert Agreement Scale. A Likert Agreement Sale is not as taxing to the survey taker as other scales, scales that change throughout the survey itself, or open-ended questions (no scale).

Survey Length. While we have been conditioned to believe long surveys reduce response rate, my real-life testing indicates that length of survey is irrelevant to response rate if the survey participants are engaged on the topic and have something to gain, the survey is professional administered to protect confidentiality, the survey is well designed, and the survey is well communicated. The overwhelming odds are that once a survey is started it will be completed. Obviously you don’t want to abuse that.

Bad question design makes people angry, and that may not be obvious to you if you designed the question. That is what testing and professional review is for!

A survey taker can scan and complete a 30 item Likert response scale survey in under 5 minutes. Don’t believe this, try it and use a stop watch. An 8-hour workday is 480 minutes, a workweek is 2400 minutes and a year is 24,000 minutes. You just asked for 5 minutes for feedback. 5 minutes is a rounding error.

If people say they are surveyed out or they have survey fatigue it means you are running a lot of bad and unprofessionally designed surveys. That is not an indication surveys are bad. That is a different problem.

Samples Not Census. I want to be clear you don’t need everyone to take the survey– you need a representative sample that is large enough to provide a confidence interval you can learn within. Be clear about your goal. Good isn’t everyone takes the survey and likes to do it. That is unrealistic and wasteful. Good is we get some people to take the survey and learned something from it. We don’t survey just for the sake of survey.

Analyze the Analysis. Over time you need to identify questions that have analytical value for purposes of measuring progress and correlation to other outcomes. And those that don’t!

If the learning produced from asking a question is not useful,

-      or the variance in response to an item is not useful,

-      or if the item doesn’t relate mathematically to an important outcome,

-      or if you aren’t going to make any changes in response to feedback,

then questions should be removed.

Reduce Redundancy. If you identify questions that people always respond to the same way then you should remove one of the redundant items even if you think the question is different and like them both.

The phenomenon I describe happens a lot when you ask about a specific person (a manager or recruiter for example). The survey respondents decides simply if they like this person or not. It doesn’t matter if you ask 10 questions about that person, whether they like that person will shape their response to all 10 items equally. We wish they carefully answered each question without any bias but wishing isn’t helpful. Don’t fight reality. You may think they are all great questions, but if the reality is that you are catching a blunt signal then conform to that reality and ask one question.

This problem is why I offer some questions that measure what a recruiter does without explicitly saying recruiter or naming the recruiter. This helps to remove that bias of whether they liked or didn’t like the recruiter as a person or fear for that person. If you don’t say recruiter or the name of the person you get a cleaner measure of whether or not important actions were completed that can provide recruiters quality feedback with less bias.

Confidential, Not Anonymous. There is a difference between confidentiality and anonymity and it is extremely important to keep them straight.

An anonymous survey means everyone using the same survey hyperlink and there are no identifiers. Therefore, it is impossible to match data at the individual level. It also means the same person can take the survey as many times as they want (or anybody else could). If you are doing an anonymous survey you must ask participants to provide all information you wish to report results by on the survey itself. If you want to cut results by Job Family, Job, Hiring Manager, Recruiter, Location, Gender, Ethnicity, Generational Cohort, other outcomes, or anything you will have to ask it on the survey itself, they have to know the answer, and they have answer it correctly. You have transferred workload and risk of error to the survey taker. Also keep in mind that whatever you forgot to ask you can never go back to get - you will never be able to report or analyze your results by something you didn't ask.

A confidential survey is an alternative to an anonymous survey that address the privacy needs of the respondent, while preserving ability to analyze the data without transferring workload or risk. Confidential means you have individual level responses, but you won’t look at individual level responses. Confidential means you agree not to report below a sample size that would expose the confidential feedback of survey taker or identify. For example, you agree to only report in groups above 3. Usually we say 5 or more but in theory 3 is enough to protect confidentiality.

Providing assurance of confidentiality is important to encourage survey-takers to feel safe providing an honest response (or any response at all). It also serves to ensure a minimum threshold of data for statistical significance, reliability and comparability.

With a confidential survey everyone who takes the survey gets a unique link or code, so you don’t have to ask survey participants for basic information, AND so you can add other data (anticipated or unanticipated) to produce longitudinal or correlative analysis including all relevant data sources.

In 20 years across companies in different industries, conditions, and sizes I have never had a problem with the response rate on a confidential survey. I stand by this – I can provide mountains of arguments why anonymous surveys achieve no advantages over a confidential survey.

Always use a third party when surveying employees, former employees, or anyone who may become an employee.

The third party:

-      protects confidentiality of survey takers,

-      demonstrates a level of professionalism we should all expect,

-      helps to address conflict of interest problems,

-      helps to build trust,

-      helps to comply with GDPR and employment law risks that having the data directly would otherwise subject you to,

-      and produces value-added cross-company perspective and benchmarking.

If I receive a survey from junksurvey.com it implies the survey was put together by amateurs that don’t know what they are doing. I will never take that survey. That survey also permanently diminishes the companies’ brand in my mind. 

To summarize: Use a third party. They join data and then anonymize and re-share with you in a pre-joined anonymous dataset you can still analyze, or they do all the reporting and analysis work you need for you. You need to make it very clear to the third party this is what you want and what you are paying for. If they want this job, that is their job, no excuses. Survey tools are commoditized and ubiquitous – you are not paying them for their survey tool. There is no other reason for their services. Make sure their price is reasonable.

Communicate explicit rules on the survey on how confidentiality is protected and how results will be reported back to the company. Rules typically say that there must be a minimum number of responses in a reporting segment (team, location, role, etc.) before data will be reported back for that segment. Usually it is 5. In some situations, I go as far as to provide and QA that precisely states how the data will be used and who will have access to the data.

Longitudinal, Re-Segmental and Correlative Data Design.

Most people imagine in their minds the result of the survey reported one time per item for the entire company or just comparing the response to each question to the other or viewing that by itself. In most cases, this one time reporting concept I just described will produce no insight. It will be a complete waste of time and it reflects a lack of professional expertise with surveys or creativity, not a problem with surveys themselves. If you collect survey data the right way then you should be able to provide new perspective and insight with the survey results. Again, in 20 years of working with organizations and surveys, I have never seen a major survey we really put our hearts and minds into not produce new unexpected and valuable insights.

What do I mean by perspective and insight?

One insight is being able to see if a measure is changing over time, the range of movement and see if the change associates with particular events or actions. That is discovered through longitudinal analysis.

Another insight is being able to see the results cut by segments that produce new perspectives. Is one pipeline performing differently than another? Is one team performing differently than another. Are women experiencing the process differently than men? Are people who are different in some way experiencing the process differently? Are we consistently performing better among the key roles for the company’s success than generic roles?

Another perspective is taking all of the questions you ask and data you have to correlate it to some other index or outcome you wish to produce. As an example, you might correlate response to all other items on the survey with the question, “How likely are you to recommend a job at xyz corp to close friends or colleagues”? You don’t just want to know that people respond negatively to an item – you also want to know if that negative response matters or not.

Questions matter to a different extents – they are not equal. You determine an area of improvement – NOT BY HOW LOW THE RESPONSE IS – rather, by confirming that the range of response correlates to an outcome you care about. Only then does a low response matter. Should go without saying, only if it matters, is it worth your effort. My manner of speaking is sometimes difficult but this is not a difficult idea. It is extremely important to understand.

Finally, even more insights can be obtained when you run experiments or A/B Tests on your recruiting process. An experiment is when you try something different on a small sample of the whole. Try a new way on 50 out of 100 candidates and measure the same things for both. If you find a better result with the 50 then you can begin to apply it to everyone. If you do not then stop!

Talent Acquisition Analytics

• “Measuring Candidate Experience”, July 17, 2018, LinkedIn.
•  “Measuring Employment Brand and Attraction”, July 14, 2018, LinkedIn.
• “Creating Competitive Advantage with Talent Acquisition Analytics”, July 12, 2018, LinkedIn.

More

• Find more of my writing here: Index of my writing on people analytics at PeopleAnalyst
• Connect with me on LinkedIn here: https://www.dhirubhai.net/in/michaelcwest
• Check out the People Analytics Community here: https://www.dhirubhai.net/groups/6663060
• Buy my book on Amazon here: People Analytics For Dummies , directly from the publisher (Wiley) here: People Analytics for Dummies , or from other places where books are sold.