The Skills Management Revolution: Appendices

This contains the three appendices that accompany the article The Skills Management Revolution: one-year, two-year, and three-year predictions.

Appendix 1.? How skills ontologies enable better skills management

Appendix 2: Skills ontology data sources

Appendix 3: Why skills ontologies may not change the future of work.

Appendix 1.? How skills ontologies enable better skills management

To understand why skills ontologies are better than previous skills management methods, it is necessary to first look at how companies historically managed skills.? In the past, companies manually created and organized skills libraries using hierarchical taxonomies with different categories of nested skills. For example, a taxonomy might have two broad skill categories called “math skills” and “programming skills”. The math category might have a subcategory of skills called “statistics”, while the programming category might have skill subcategory called “SAS language”. Because SAS is a statistical programming language, a person who has skills in SAS probably has skill proficiency in statistics. But a traditional skills taxonomy would not recognize this since SAS is categorized as a programming skill and not a math skill. What skills ontologies do is identify relationships between skills without relying on hierarchical categories. For example, from analyzing thousands of resumes the solution might infer that people who have SAS programming skills also tend to have statistical analysis skills. The solution might also infer from career experience data that people with statistical analysis skills are often successful when hired into jobs that require learning SAS. The skills ontology recognizes that statistical analysis and SAS are related even if this is never explicitly stated. The ontology can infer that someone who has SAS skills almost certainly has some level of statistical skills. And that if someone has statistical skills then they are likely to have an aptitude for learning SAS.

The concept of skills ontologies has been around for decades. But it was not possible to replace skills taxonomies with skills ontologies due to the work required to map relationships between skills. The recent move to skills ontologies is a result of three technological advances. First, the accumulation of masses of digital data about skills in the form of online job descriptions, job posting, resumes, individual profiles, self-assessment questionnaires, electronic communication such as e-mail and chat board postings, interpersonal relationship networks, training course descriptions, and other source of information that can be used to infer the skills that companies need or the skills that people possess. Second, the development of natural language parsing technology that enables companies to collect and interpret this information automatically. Third, the creation of machine learning technology that makes it possible to automatically create and update skills ontologies using different sources of data[i] .

Modern skills ontologies incorporate millions of data points to infer individual skill levels by looking at interactions across thousands of data sources. For example, an ontology might deduce that a candidate is likely to have some level of project management skills based on their work history even if they never listed project management as a skill, but instead shared they were the entertainment director for their sorority in college, worked for several years as a retail store manager, and are currently the president of a volunteer children’s theater association. The ontology might recognize from other data sources that all of these activities reflect some level of proficiency at managing projects.

Most of the work required to create ontologies is done through computer algorithms with little to no human involvement.? However, development of ontologies is not entirely automatic.? Subject matter expert guidance is critical to determining what data to feed into the ontology and to review and refine inferences about skills that ontology may draw from this data. ?However, the level of human effort required to create modern skills ontologies is a fraction of what was required to create traditional skills taxonomies. ?And the insights provided by these ontologies is vastly superior to what was possible using more manual and hierarchical skills management methods.

Appendix 2: Skills Ontology Data Sources

A key feature of modern skills ontologies is their ability to import, interpret, and organize massive volumes of skills data from a wide range of sources.? Any form of data that might reflect skills a person has or skills a company needs could theoretically be fed into the algorithms used to generate skills ontologies. The breadth and quality of the data used to build the skills ontologies is a critical component affecting the performance of modern skills management solutions. This includes company specific data sources, which often play a particularly important role in building effective ontologies.

The tables below lists types of data that could be used to create a skills ontology, why this data is useful, and potential concerns about its use. The data sources are grouped into four categories.?

• External Talent Data: External data about the skills of individuals in different labor markets contained on public sources such as online career networking sites.
• External Market Data: External data about skill trends in the marketplace contained on public sources such as job postings, government labor statistics, news articles, and community boards.
• Internal People Data: Internal data about employee, candidate, and contractor skills from company sources such as employee profiles, job applications, talent reviews, and training certifications.
• Internal Company Data: Internal data about organizational skill requirements from company sources such as business strategy documents, job descriptions, or learning management catalogues.

Most skills ontologies use some subset of the data sources listed in this table, although it is unlikely any ontology would use all of these sources. Data used by skills ontologies tends to change based on whether it is designed to support workforce planning, recruiting, learning, internal talent development, or some other skills management activity.? This list is also not comprehensive.? There are a great many sources one could use to learn about skills relevant to different kinds of jobs or industries. ?Knowing which data to include, being able to access it, and effectively cleaning it for analysis is a critical part of building skills ontologies.? It is also important to be aware of data security and privacy legislation that might limit the ability or desire to use certain types of data to build ontologies. ?In addition to focusing on what data should be used to build an ontology, it is important to consider what data should be excluded.? Companies and/or employees might have concerns over use of certain kinds of skills data, particularly if it is being used to influence “high stakes” talent decisions related to hiring, compensation, and development.

Appendix 3: Why skills ontologies may not change the future of work – a historical case study.?

I suspect skills management ontologies will change how organizations manage talent.?But one could argue that skills management ontologies are unlikely to change things very much. ?And there is a historical precedent to support this view. Although this paper takes a different perspective, the scientist in me feels compelled to offer this alternative hypothesis.?

The primary purpose of most skills management solutions is to improve staffing and development decisions. In this sense, the growth of these solutions is reminiscent of the increased use of personality assessments that occurred from around 1995 to 2010. Personality tests were widely used for staffing and development in the 1950s and 1960s. Then research called their accuracy into question, and companies largely abandon their use during the 70s and 80s.?Three things happened in the late 90s that changed this: 1) advances in psychometric science led to more accurate assessments; 2) the shift to a knowledge and service-based economy increased the importance of “soft skills” for job performance; and 3) the internet made it far easier to use assessment tools. Initial use of personality tests focused on improving employee selection and development.?But as use of personality testing grew, some people talked about rethinking work itself based on personality attributes. Test vendors envisioned comprehensive personality models that could be applied across all aspects of work.?Master personality profiles would be used to redesign teams and jobs to complement personality characteristics of employees. None of these transformational changes materialized. Instead of a few comprehensive personality models, there are now hundreds of assessments designed for specific applications such as sales, leadership, customer service, and career guidance. And the concept of reimagining job and team design based on employee personality never happened except in a very limited sense in a few companies. Personality tests are a valuable part of modern staffing and development programs. But personality testing did not radically change how organizations manage their workforce.?It just made companies better at doing things that they were already doing.?The same could happen with skills ontologies. They might just end up being useful tools to improve workforce forecasting, staffing and internal talent development.?Rather than reshaping the world of work, they will simply become a minor but valuable part of the HR technology landscape.

The degree to which skills ontologies are utilized to create skills-based organizations is likely to hinge on the ability to convince employees and leaders that these ontologies provide accurate, fair, and valid insight into employee capabilities and potential. As a general rule, people do not like to be judged solely by mathematical algorithms. People reluctantly accept that other people may make imperfect talent decisions that impact their careers but have far less forgiveness when machines misjudge their value and potential. ?Before we see widespread adoption of skills ontologies to guide workforce management decision skills management vendors will need to address one particularly critical topic, which is the same core topics that undermined widespread adoption of personality tests: assessment validity.

Skills influence job performance, but that does not mean that the skill profiles created by ontologies are good predictors of performance. Making hiring decisions about a person based on their skills ontology profile is like making hiring decisions based on looking at someone’s online resume. There are reasons why skills profiles are likely to be good predictors of job success, but there are also reasons why they might provide misleading data about a person’s true performance potential. This is particularly true when assessing skills where there are substantial differences between novice and expert performance.? ????

The predictive validity of matching scores based on skills ontologies needs far more study if these solutions are going to become central to workforce management. Just because a skills engine made a recommendation for who to hire or what training someone should take does not mean it was the right recommendation.? We need to better understand the quality of these recommendations.? This will require investigating three things.? First, whether skills profiles created about employees are accurate reflections of their actual skill proficiency. Second, whether job requirements based on skills ontologies accurately capture the skills needed to be successful in different roles. And third, whether talent recommendations based on skills profiles accurately predict future success associated with job performance and/or learning capability.

At some point candidates and employees who did not receive the recommendations they wanted or expected from skills ontologies will ask, “was this fair”? ?And business leaders will wonder “are these solutions worth their cost compared to other ways we might management talent?”.? Skills management vendors will need to respond to these questions with defensible, clear, and comprehensible evidence-based answers that show their solutions truly are doing what they are claiming to be doing: optimizing workforce costs while helping people realize their full potential.?

[i] One of major factors that has enabled use of skills ontologies is increased computer processing speed. This makes it possible to build ontologies using machine learning and natural language parsing algorithms. In 2005 I was involved in building a skills ontology for an early internet recruiting system. While conceptually it was similar to modern skills solutions, the effort and time required to maintain it rendered it impractical in application.?