In Defense of Middle Measures: the Use of Constructs in HR Analytics

In Defense of Middle Measures: the Use of Constructs in HR Analytics

I've read a few HR analytics articles lately asking about the predictive power of “middle measures” in HR analytics.  "Middle measures" in this case being defined as measures that are not directly observed, but potentially still responsible for employee behavior.  I believe a lot of the conversation has been spurred by a much needed debate around employee engagement*. Engagement is an abstract middle measure that many analytics companies use in models without fully understanding where it comes from, but more on that later.

Most HR analytics initiatives get started with the directly observable variables like demographics, position data (function, job family, role), and tenure because this variables are easily accessible from current HR data warehouses.  However, there's a hesitation to pull in the calculated results of survey data or more complicated human psychology variables since they are "middle measures" that root from directly collected data. Instead of “middle measure” I’d like to recommend the term “constructs” to drive this conversation forward. 

Construct is a term that a lot of I/O psych and social science readers out there will recognize.  As predictive analytics turns towards HR as the next source of optimization within the business, I want to pull this term from the social sciences because using it will allow the field to better access the associated academic research. By thinking about these abstract measures as constructs we can tap into and quickly apply the thousands of PhD hours spent studying these concepts. 

*For a brief rundown of why we should take Engagement with a grain of salt please see Rob Briner's thoughts on the subject. 

What is a Construct?

With regard to predictive modeling, there may come a time where we want to include some human psychology / personality variables in our models, but when we go to add them we’re unfortunately not able to directly observe them.  For example, it would be great to have a “job satisfaction” variable in the model for predicting employee attrition, but we can’t pull out our “satisfaction thermometers” and record it's values (and from an HR perspective I don’t want to imagine that consent process). If we can’t measure job satisfaction directly, we need to come at it from another angle. 

There’s a quote from George Box that I think applies well here which is “All models are wrong, but some models are useful”.  While we can’t measure personality traits directly we can essentially make a model of them using known observable behaviors such as “attendance” / “tardiness” or a collection of responses to a survey.  We can then use that model which is based on the direct observations to predict a measurement of the more abstract variable of job satisfaction.  That measurement we arrive at is the construct representing job satisfaction.

Once we have that construct (measurement), we can then use it as a variable in other models as a predictor for employee behavior.  Measuring people is incredibly complicated, but workforce models become more accurate when we can include psychological measures such as “organizational commitment”, “intent to leave”, or “job satisfaction” to name a few.  The beauty of the construct is that if the construct is well-designed, it will have greater accuracy than if we had just used its directly observable components.  

How do you define a construct?

Constructs start with a theorized conceptual definition and then are built out with operational definitions that explain how we arrive at a value for the construct.  This is a process I feel HR analytics (or People Analytics, Workforce Analytics, Talent Analytics, etc) teams should steal and put to use.  As an example, the most widely used conceptual definition of job satisfaction is:

“A pleasurable or positive emotional state from the appraisal of one’s job or job experiences” – Locke (1976)

Job satisfaction is one of the most widely studied constructs in managerial science (thousands of papers have been written on the topic) and as such researchers found the need to work from a common definition.  This particular definition was developed by Edwin Locke of the University of Maryland in 1976 is one of the most widely used.  

Now, when researchers from any university begin a study on job satisfaction, if they establish this as the conceptual definition then their work can be blended into the larger work done by the field.  If there was not a common definition, work would not be able to be referenced or reproduced outside of a given study without redefining the construct and starting over.  This unfortunately seems like it will be relatable for HR analytics teams.  Without a central system of definitions, every company and sometimes every team is on it's own to define their constructs.

An operationalized definition takes the conceptual definition, and lays out a plan with which to measure the construct.  For the example of job satisfaction, there has been a massive scholarly effort to rigorously test and perfect different operational definitions.   For example purposes, here is the leading operational definition of job satisfaction:

Job Descriptive Index (JDI)

Measures job satisfaction using 5 facets: Work on Present Job, Present Pay, Opportunities for Promotion, Supervision, and Coworkers.  Each facet contains either 9 or 18 items.  These facets can give organizations a hint at which aspects of the job need improvement and which are in good shape.

The JDI index is not only the definition above, but an associated survey, manual, scales, and scoring method.  It is owned and maintained by Bowling Green University who makes the index available for FREE at this link: Job Descriptive Index.   If you've read my other article on R programming, you'll know I love free.   

Even more than free, I love open source HR and there are few things in our space more open source than the associated academic research.  There is a rich history of management science around many of the areas that HR analytics practitioners are currently studying.  Instead of reinventing the wheel on job satisfaction, we can and should tap into these operational definitions to put to use measures which have been vigorously validated by academia and then launch our studies from there.

What makes a strong construct?

To qualify as a strong construct, a measure needs high test reliability and validity.  Test reliability is in essence the repeatability of the test that measures the construct. Referencing job satisfaction again, that measure is highly reliable.  It has been clearly defined by the definitions stated above and, along with the JDI manual, rolling it out as a measure in any given company should not be a problem.  No matter who gives the test, what form they take it in, or who measures it, it produces the same results based on participant answers. 

Validity then refers to how well the construct represents the underlying abstract measure.  Validity of a construct like job satisfaction is best understood in terms of predicting an employee behavior, like attrition.  Job satisfaction when used as a predictive measure for attrition has varied results depending on the study. However it is largely agreed that it is a valid construct which predicts, on the low end, 3.6% of the variance in attrition (Griffeth Hom 2000). 

That might not seem like a powerful explanatory variable to someone outside of the social sciences, but 3.6%, while low, is incredibly helpful when it comes to explaining human behavior.  It’s a concrete measure within the managerial sciences and worthwhile to include in a model of turnover.  For a deeper and more rigorous review of how to test a construct's reliability and validity, please check out the following links: Validity and Reliability.

Questions to Ask

I want to leave you with questions to step through when you are thinking about designing a construct or presented with a construct from a vendor.

1.  What is the conceptual definition?
2. What is the operational definition?
3. Is it able to be measured repeatedly (reliability)?
4. What is the construct predicting (validity)?

In the reading above you've seen me walk through this for job satisfaction, a highly studied and well constructed measure.   Try to apply these to a few measures that you use on a daily basis and see if they hold up.  I'd particularly love to hear your thoughts on applying these to a more tenuous measure such as Engagement.

-----------------------------------------------------------

I hope I've left you with something to think about today!  I'm working on a few more articles bringing academic work to the practitioner space, so if you enjoyed this piece please follow me to catch the next ones as well.  I'm looking forward to hearing your thoughts on these articles and the blog in general, so please reach out to me here or on Twitter with any comments, suggestions, or questions. The support and community around these topics has been awesome; thank you for reading.

- Richard Rosenow

 https://www.dhirubhai.net/in/richardrosenow

https://www.twitter.com/RichardRosenow

 My other posts:

• Analyzing Employee Turnover - Predictive Methods
• Analyzing Employee Turnover - Descriptive Methods

• HR Analytics Starter Kit Part 1 - Intro to HR analytics
• HR Analytics Starter Kit - Part 2 - Intro to R programming 
• HR Analytics Starter Kit - Part 3 - Podcasts
• HR has Last Mover Advantage in HR Analytics