The validity of the TR safety observation method on building construction

This 1999 study evaluated the performance of a construction safety field observation method – called the TR safety observation method (‘TR’ is said to be an acronym for building construction in Finnish).

I think this may interest people given the popularity of field-based critical control observation processes – you may even point out similarities with behavioural sampling approaches.

Background:

·???????? “A construction site is a more demanding object for reliable observation than the manufacturing industry because the work environment changes continuously”

·???????? The TR method studied here started back in 1992 in cooperation with the Finnish Institute of Occupational Health, a safety inspector and a big construction company

·???????? Their goal was a standardised observation process that gave reliable results at any building construction site, and applicable across construction stages

·???????? It also had to be simple enough for routine use by site personnel

·???????? Safety inspectors carried out field observation monitoring at 305 building construction sites, and results were compared with the site accident figures

·???????? The average number of observations per site was 144

·???????? They assessed safety aspects related to working habits, scaffolding and ladder use, machine, fall protection etc as correct or not correct

·???????? A safety index was calculated as a percentage of ‘correct’ observations

·???????? They argue that “The use of accidents as a safety indicator of a single building construction site is in most cases impossible. The construction site typically lasts for only 1 year, and there are on average only about 20 workers on each site. The average accident rate for building construction workers in 1995 was 129 accidents (work absenteeism of at least 3 days) per thousand man years (Statistics Finland, 1998).

·???????? Hence, “These figures mean that only two accidents on average occur per one site. Because of random variation, many sites have no accidents and it is not possible to say whether they are safer than other sites with four or five accidents”

They describe the TR method:

·???????? The observer walks the whole site and takes a sample of at least 100 observations to get a representative sample

·???????? Observations are marked as strokes on a one-page observation form which contains not more than six items to be observed

·???????? The observed items must include the most significant accident risks, including the severe but infrequent ones

·???????? The most important safety items are observed more frequently than the less important ones, and given more emphasis in the safety index and hence “use of separate and complicated weighting coefficients is thus unnecessary”

·???????? Observable items include working habits, like use of protective equipment, scaffolding and ladders, machines and equipment, protection against falling (guard rails, protected penetrations etc.), and more

?

Results

Based on the TR method, they found:

·???????? “There was a significant correlation between the observed safety index and the accident rate of the site groups”

·???????? “The sites with the lowest observed safety index had, on average, a three times higher accident rate than the sites with the highest safety index”

·???????? The validity of the method “proved to be good” and the “interobserver reliability of the observation method proved to be good enough”

They found that the average safety index of the sites was 67%; the lowest index was 21% and the highest 95%. The distribution of the 305 sites according to the safety index followed a normal distribution where “Most of the sites located in the middle of the safety scale, and there were fewer sites at both tails of the scale”.

They observed that the best performing sites regarding the safety index performed well in all aspects (>85%), whereas the poorest performing projects performed poorly in all aspects.

Nevertheless, “Protection against falling and order and tidiness were the weakest aspects in all the groups”.

They found that only 2h of training in the method was necessary for acceptable interobserver reliability.

Importantly, “ On a single building site the observation method is even a better indicator of safety than the accident rate”, notably because accidents are statistically rare.

Moreover, “Based on accident figures only, one would say that more than half of the sites were absolutely safe because they reported no accidents at all. But this is impossible. This study showed that there have to be dozens of sites to get reliable accident figures”.

And “There is every reason to be suspicious of research conclusions that are based on the accidents figures of only one or a few building construction sites. The observed items in the TR method cover all the significant direct causes of both slight and serious occupational injuries in construction” (with the exception of over-exertion and biomechanical hazards).

Given the connection between the observed factors and the risk of falls, they “had every reason to assume that the observation result is also in this regard a better indicator of safety than is a small sample of accidents”.

That is, while the researchers used accidents as one proxy of the success of the method, they also note the inherent limitation of using accidents (because of their statistical rarity), and hence, argue that observing the precursors to accidents, as like this TR method does, is a more robust approach.

Likewise, perhaps we could (weakly?) infer that similar approaches focusing on precursors and direct controls may be similarly sound?

Authors: Laitinen, H., Marjam?ki, M., & P?iv?rinta, K. (1999). The validity of the TR safety observation method on building construction.?Accident Analysis & Prevention,?31(5), 463-472.