Does sending Safety Toolbox Talks by text message to Residential Construction Supervisors increase Safety Meeting Compliance?

This US study evaluated whether distributing construction safety toolboxes, with workplace fatalities, to supervisors by mobile phone would increase their compliance delivering toolboxes each month,

56 construction supervisors in Oregon were recruited and received a link to a toolbox talk by text message every two weeks for three months. This is based on a tool called the OR-FACE (Oregon Fatality Assessment and Control Evaluation).

Providing background:

·?????? They say that use of project-specific training and safety meetings has been linked to being “

·?????? In the construction industry, having project-specific training and safety meetings has been identified as “one of the three most commonly adopted safety “innovations” in the construction industry over the past four decades”

·?????? Moreover, participation of contractors in safety meetings “has been identified as one of 14 differentiating safety practices of construction projects with lower experienced recordable injury rates”

·?????? It’s argued that perhaps use of structured guides may further improve the quantity and quality of leader-member exchanges in toolboxes – enhancing particular communication/engagement behaviours and facilitation in such engagements

·?????? Nevertheless, despite the importance of engagements, “most interventions in the construction industry focus on reducing injuries or improving safety knowledge as the primary outcome, with mixed results”

·?????? Despite the importance of communication and engagement, proportionally less research has been undertaken in construction, at least in residential construction

·?????? The OR-FACE program developed scripted safety toolbox talks to “help supervisors share fatality stories and discuss prevention recommendations in safety meetings”

·?????? Using this approach, toolboxes are around 350 words in length and based on fatalities, in a 2 page format, with the second page containing dot-points and simplified line drawings to be shown to workers

·?????? Use of these prepared toolboxes were shown to reduce the safety meeting preparation and presentation time by 15 mins compared to other types, and “much of that time saved was spent in safety discussions with crew members”

·?????? Prior research has shown that “reducing response effort”, such as via pre-preparing toolboxes, may reduce “response effort for new or complex tasks”

Results

Key outcomes were:

·?????? Compared to baseline (where 34.5% of supervisors met or exceeded at least one toolbox per month), after the intervention period 48.39% of supervisors met or exceeded the minimum standards; representing ~19% improvement

·?????? This effect wasn’t statistically significant, however

·?????? Perceived communication quality didn’t significantly change from baseline

·?????? Supervisor-rated employee safety compliance also didn’t significantly change from baseline

Interestingly, although the paper suggests that there was a ~19% improvement in compliance post-intervention, they also recognise that this change wasn’t statistically significant. So, we don’t have evidence that the intervention was different to chance alone. They talk about an “encouraging trend”, but this is problematic statistically speaking.

They provide some reasons why the results weren’t statistically significant. One, there may have ben a ceiling effect, where already highly rated safety communication quality had little room for improvement (already rated at 4/5 pre-intervention).

They also couldn’t assess the workers responses. They provide several other explanations, including limited sample size [** but other research I’ve read suggests that smaller samples may make a spurious significance result more likely, rather than less likely.]

They suggest that while “the routine occurrence of toolbox talks may influence individuals to comply with safety requirements”, the routine toolboxes ”may not provide motivation to engage in more discretionary safety behaviors”.

Moreover, qualitative comments collected during study “suggested that many talks were somewhat irrelevant for their current job tasks, indicating a desire for more tailored talks to deliver to their crews during specific construction stages”.

In all, this study didn’t evidence that the use of a mobile text-driven toolbox approach statistically improved the number of toolboxes, nor the quality of content, but the authors suggest there may be a "promising trend" [** take with a grain of salt…]

Link in comments.

Authors: Rice, S. P., Rimby, J., Hurtado, D. A., Gilbert-Jones, I., & Olson, R. (2022). Occupational Health Science, 6(3), 313-332.