With Peers Like These…, Part 7: Research? Really?

During the past weeks, my newsfeed presented a couple of shares of articles saying, “Young workers are ignoring health & safety”, or something similar. These news items drew on a white paper, claiming to be based on research done by a firm specialized in mobile solutions “to capture all manner of data on PCs and mobile devices in real-time, providing a more accurate and efficient alternative to paper forms”.

I have no experience whatsoever with their products, but I am happy to believe that this firm delivers great tools. Which does not imply that they are also able to deliver great research, of course. Or that they deliver research at all. Let us take a critical look…

Something fishy?

They say we should not judge a report by its title, but it is somewhat odd that the title is ‘The Future of Health of Safety’ when it is mostly backward looking. Whatever. The authors need to draw attention since it appears that they have a serious message to share with us. After all, it is “A study describing how almost two thirds of UK businesses are failing to meet basic health and safety laws”.

Now, let that sink in. Heavy stuff. Stuff that made me very suspicious. Two thirds? Really?? One of many passages that smelled a bit ‘off’. On page 11 of the white paper, its conclusion states, “Overall, the current state of health and safety in the UK is certainly below an acceptable standard”. This may have been true in Dickens’ time, but does not correspond with my experience.

Also the Health and Safety Executive (HSE) appears to have a different opinion: “Great Britain’s health and safety record is the envy of much of the world. Central to this is protecting people by managing risk in a proportionate and effective way, supporting innovation and increasing productivity. The challenge is to improve even further on this impressive record”. Hmm, weird.

Testing time

When encountering make-belief research, often a few basic checks are sufficient to get a sense of its value:

1. Check the statistics and the design.
2. Check facts.
3. What are the premises and assumptions?
4. Who benefits?

1: Stats and Design

The white paper tells us that a survey asked 2.000 employees working in various businesses with more than five workers about their health and safety procedures. Apparently, the results from this survey were extrapolated to an entire nation, which currently has roundabout 32 million people in employment. I have not done the math, but I wonder how valid this is, statistically.

Besides, what about randomness and representativeness? One cannot tell for sure from the white paper, but were the data collected with the tools of this firm? Then we must assume that the results are not representative of the UK workforce.

2: Cherry Picking

It is not always possible to verify facts, but since the white paper appears to draw on some HSE statistics, it is easy this time. The white paper mentions an increasing amount of fines and a 6% rise in prosecutions from 2015 to 2016. Indeed the number of prosecutions appears to increase steadily since 2011/12 (Figure 1 in the HSE report). Whether this is caused by a deteriorating state of health and safety (as the white paper claims), or by other factors, like new policies or political goals, or increased building activity and other context factors we cannot read from the HSE report. We may actually wonder how relevant the increase is, regarding the absolute numbers.

As we can read from the HSE report, the bulk of the increase of the money involved in fines is due to 14 fines that were higher than the maximum fine imposed in the previous years. The white paper apparently forgot to mention that. Just like they ignored the total number of enforcements notices that appears to be on the way down since 2011/12 (Figure 4 in the HSE report).

We all love cherries, don’t we?

3: Safety = Procedures

The white paper claims to have exposed a bad state of health and safety. But have they? Look at what they ask, and how they frame their results. While the white paper claims to be about improvements to protect workers, all it talks about is rules, procedures, rule breaking and compliance.

What you ask for is what you will find. After that, all you need to do is take a few short cuts. From not updating procedures and not reading the handbook to the verdict ‘unsafe’ and you have the making of a report with alarming results.

Yes, safety rules are important and a basic feature in safety management. But they are not the only element and they have serious limitations. Therefore, the white paper’s conclusion requires some serious modifications…

4: Follow the money

At this point, it may not come as a huge surprise that there is a silver bullet for sale. Buy a tool that enables you to do a digital, easy to update manual and your health and safety is covered. You just need to apply another couple of shortcuts, because providing employees with the latest legislation is not the same as ensuring that they operate compliantly, nor does compliant mean safe.

When reading ‘research’ always check if the authors have a stake in the outcome. If the answer is yes, be very sceptical.

Conclusion

Fail on all four check points. Sorry, this is not research; this is merely a very thinly disguised sales brochure. Pity that many fall for it nevertheless. I can understand that laypersons would do so, but I have to urge safety professionals, who are supposed to have the baggage on board to see through it, stop sharing nonsense like this. It is perfectly all right to advertise for a product, but please be open about this and do not try to hide it under a pretence of research.

---

Carsten Busch is a self-declared Safety Mythologist and author of the well-received book Safety Myth 101. This book collects 123 (and then some) of Safety Myths. Crisp and compact discussions address weaknesses of conventional safety ‘wisdom’ and give suggestions for alternative approaches and improvement. There is a section about basic (understanding of) science in there, but alas the book will not help you doing basic math. It might, however, help you being critical to just about any make-belief research that is tossed in your face.

https://www.mindtherisk.com/the-book