Metrics: Everything and the Kitchen Sink

Total Recordable Incident Rate (TRIR) has been the standard metric used by the safety profession to measure safety performance in many industries. The Construction Safety Research Alliance (CSRA, 2020) reviewed 17 years of data involving an investigation into 3.2 trillion work hour and found no discernable association between TRIR and fatalities. Further, the CSRA (2020) data analysis also indicated that the occurrence of recordable injuries is almost entirely random.

Historically, if an organization’s TRIR is better than the industry standard, the organization has been thought to have better safety performance and is usually rewarded with the award of work with partners seeking a high focus on safety. If an organization’s TRIR is below the industry standard, the downside is not being awarded work or other bonuses on existing projects.

OSHA also cares about TRIR. The reason OSHA cares about TRIR is because it analyzes trillions of hours of worker data to get a bigger picture of injuries and illnesses across all industries. Looking at an overall TRIR across all industries is valid; however, the TRIR becomes less relevant when we drill down to the detail needed to assess safety performance as the individual organization or specific work type.

Recent research in safety science indicates that TRIR is not the most accurate reflection of the capacity or performance of the overall safety management system (SMS) for a company is working at peak performance. What then is a suggested approach to metrics that provides both reflective and predictive aspects of managing safety performance, lowering injuries and illnesses, and preventing serious injuries and fatalities (SIFs)? There must be a better way to measure the presence of safety and work toward the absence of risk.

Multiple Lines and Levels of Evidence

Multiple lines and levels of evidence (MLLE) is a systematic approach to causal inference that involves bringing together different types of evidence (lines of evidence) and considering the strength of the evidence in terms of different indicators of a causal relationship (levels of evidence). MLLE involves more than just triangulation of different types of evidence. It involves:

·???????Lines of Evidence

Different types of relevant evidence, whether existing or new data.

·???????Levels of Evidence

Strength of support for a causal relationship, not in terms of a hierarchy of evidence but in terms of different indications of a causal relationship, such as temporality or specificity of association.

·???????Systematic Process

Process for judging the credibility of the evidence and the causal analysis, often involving an expert cross-disciplinary panel review of the process.

Complexity is inherent in the use of the systematic approach of the MLLE model. And, as a colleague of mine likes to say, the complexity and the inputs – in this case as the metrics relate to safety – “just depend.”

Selecting “New” Metrics

Committing to an MLLE systematic approach to injury and illness data analysis for organizations, what is the first step toward moving forward? To discover the usefulness of new metrics and combining more traditional metrics. For each metric considered for the new “equation” of SMS health, it is important to examine each metric asking the following things:

·???????Can it tell us something about the future? (PREDICTIVE)

·???????Is what I am measuring today the same thing I measured yesterday? (VALID)

·???????Is it quantitative and not opinion-based? (OBJECTIVE)

·???????How easy is it to understand? (CLEAR)

·???????Can we do something about it? (FUNCTIONAL)

·???????Can we apply enough of our time and resources to dealing with it? (IMPORTANT)

Not all potential organizational metrics will touch perfectly on all these points, but they should have quite a few of these attributes in common.

Sources of new safety-related metrics in an organization are all around us and – in most cases – specific to the exact business operations, services, and work products. Should we be providing numbers from our work task specific risk assessments? Things like times to plan, setup, complete, and repeat tasks can be valuable in determining whether rushing may cause an inflection upward in injuries or risk. Metrics that are applicable to safety can be wide ranging – from onboarding statistics, hours, and content from Human Resources to actual figures for incidents and injuries from the work site/facility to implementation of hazard controls and for how long in a process.

Your specific metrics will be the ones that you weave together at your organization to tell the story will be unique. You cannot just use TRIR anymore. Safety professionals need to think beyond safety and into the system of the organization and all its potential influences. Like an alchemist, your quest for the “right” way to measure and track safety performance will require that each building block of the story toward continuous improvement must serve a purpose – either to illuminate an area of work that needs improvement or risk assessment, or to provide support for assumptions based on one line or level of evidence only.

Suggestions for Future Focus Areas

??????????????We are aware as safety professionals that we must do better than TRIR. Looking to the future, we may be well served to use our risk assessment methods to categorize hazards in a manner that allows for higher hazard work to seek risk and predictive based metrics coupled with return on investment for hazard controls (accounting for numbers of injuries reduced and dollars saved). Software as a Solution (SaaS) companies are becoming more common and have evidenced some unique algorithms for injury prediction, risk, and actionable items for risk reduction.

??????????????One thing is certain for the future – TRIR can not continue to hold organizations in a holding pattern that does not support continuous improvement and deep introspection of our clear and present problem in safety – SIFs. To lower SIFs, we need to do better, think deeper, and expand our ideas on what to pull together for metrics to support continuous growth of our SMS within our companies.