Software Driven by Science

I have been employed by technology companies for 20 years at the intersection of software and science, albeit most times with science taking the back seat. I was always on a quest to find the right blend of technology and science as this intersection creates the best and longest lasting products (e.g., automobiles). During the development cycle it might not feel like the future is promising as traditional ways can be threatened and change hard to sell and consume (e.g., early Tesla days). Six years ago, I finally found the company to let me make the right blend of IO Psychology (science) and HR software to hopefully create a better and fairer way of hiring people. I can absolutely state in my current role at HireVue, our core mission is to have our software solidly rooted with science. In this article I am going to use some lessons and practices I have learned while at HireVue to offer guidance on how to create Software Driven by Science.

HireVue was founded by Mark Newman in 2004 with the mission every person should have equal access to, and consideration for, job opportunities regardless of their background or biased classifications (e.g., race). The first versions of HireVue’s video interview platform were designed such that standardized and structured interviews were provided to all candidates for the job. Mark didn’t know about the field of Industrial-Organizational (IO) Psychology at that time, but his design of the video interview platform mirrored best practice research from IO Psychologists on the design of behavior-based interviews. In fact, 100 years of scientific research in the field of IO Psychology, has found structured interviews to be one of the best and fairest predictors of job success.

Almost two decades later, HireVue software is still blending and being led by different science-based disciplines. About 6 years ago, the growing adoption of HireVue’s video interview platform actually created another problem for their user base – easier access to apply and interview for jobs created applicant volumes recruiters and managers couldn’t handle. Needing to review 1,000s of interviews for a single job opening created significant process inefficiencies which meant many applicant interviews were never viewed and summarily rejected (an identical problem with resumes by the way). Even with lower volumes the same problem exists as many applicants weren’t being evaluated because they entered the applicant pool to late in the hiring process.

Cue the scientists from various fields with Doctorate degrees in Physics, Chemistry, Bioengineering, Computer Science, Linguistics, Psychology etc. to help solve these problems. In general, the scientists at HireVue consist of two connected groups: Data Scientists and IO Psychologists. Data Scientists work on the artificial intelligence applications, IO Psychologists work on the psychological profiling of applicant data, and together they solve the problems of extracting meaningful predictions from large, diverse data sets in ways that drastically improve human decisions. It is clear HireVue has created better HR-technology by having eclectic scientific viewpoints. 

The best way to describe the Charter of the Science team at HireVue is to enable human decision making that is highly predictive of job success and significantly less biased than if the decisions were made without the science. So how do we provide the “scientist in the box” for our customers? In general, we have figured out a way to “score” an applicant’s video interview response or game play behavior using language analytics, audio patterns, and behavioral cues. For video interviews, we score an applicant’s responses with our assessment algorithms in the same manner an interviewer would score a structured interview response, but without the live interviewer requirement. For game assessments, we score an applicant’s behavioral patterns and cognitive abilities by mapping their game decisions and patterns of play with grounded psychological theories.

 So enough about the background and theoretical stuff; let’s talk about the ways in which HireVue’s Science team is actually writing the playbook for Artificial Intelligence (AI) based HR systems – Software Driven by Science:

Science Values Driving Software: In February 2019, HireVue crafted and released the first ever AI Ethical Principles specifically for HR, although we feel they are perfectly applicable to any AI business use case. We recognize the impact our software has on individuals and on society, and we take this responsibility with deep commitment. On a daily basis, our Principles guide our science as we develop AI-based technology and tools. In fact, the below work is a result of following our AI Ethical Principles.

Science Research Driving Software: Our Science team continually evaluates our algorithms or assessments for fairness and psychological measurement accuracy. For example, across multiple studies we used 1,000’s of video interviews to fine-tune our data predictions to a point the language and audio data sets (what a person says in an interview and how they say it) provided all of the psychological measurements we garnered from a video interview. The non-verbal data points or data from analyzing the facial movements during the video interview (facial action units) did not add any value to the psychological measurements. Also, other scientific research showed the accuracy of facial analytics are degraded by darker toned skin, thus facial analysis can add to biased decisions. Given our, and other scientists research, HireVue decided to not deploy facial analysis data points in our assessment algorithms. A point by the way that was completely misrepresented in the recent HBO Max documentary called Persona.

Science Transparency in Software: Black box software is not a good thing for society, certainly not for HR applications, and goes against the principles IO Psychologists follow. Following this belief, HireVue’s Science team activated four audits of our science enabled technology; all conducted by independent experts. 

• Audit of AI Technology: This audit is a detailed analysis of our AI technology and algorithmic bias. Recent software changes to the candidate user experience to increase transparency are a result of this audit.
• Audit of IO Psychology Science:  This audit is a detailed analysis of the psychological measurements and job fit frameworks used in our AI-based assessments.
• Audit of AI Procedures: This audit is a review of the consulting procedures and design controls we place on our software when using AI-based assessments.
• Audit of Methods for Measuring AI Bias: This audit analyzed our AI-based assessment data and how we measure and mitigate bias in our algorithms.

Have a Clear Objective: Remember, science can be the death march of a new technology – try to avoid the “decimal dust” if at all possible. This means science perfection at the expense of ‘why’ and ‘how’ the technology gets productized or used can be detrimental to future success (any Segway owners reading this). At HireVue our multi-disciplined scientists blend new (e.g., machine learning) with well established (e.g., psychological measurement) techniques to create scientifically sound software tools that aid our customers HR decision making processes. All of these advancements are productized and monetized based on sound enterprise software practices.

In sum, I hope this article provides a little insight on how to blend HR-technology with science: develop your guiding principles, know the benefits and limitations of your science, design for your ultimate objective, and get views from outside your forest. This philosophy is rooted in the core of HireVue from its inception to this very day – for more information visit our science page.