The Total Recordable Incident Rate, or TRIR, is a standard that is used by most safety professionals to measure safety performance, no matter what industry they may be in. But is it statistically valid?
Research conducted by the University of Colorado Boulder involving an investigation into 3.2 trillion hours of worker data found no discernable association between TRIR and fatalities.
Further, the analysis also indicated that the occurrence of recordable injuries is almost entirely random.
An important note on OSHA
One thing to remember when looking at this information: OSHA cares about TRIR. The reason OSHA cares about TRIR, and why it should continue to care, is because it analyzes trillions of hours of worker data to get a bigger picture of injuries and illnesses across all industries. This is a valid use for TRIR.
However, the research shows that TRIR isn’t really relevant when looked at from the level of an individual company or business unit.
For OSHA it’s relevant. For the rest of us, we’re going to have to come up with a better method.
A number that stubbornly refuses to change
To measure TRIR, you collect how many recordable incidents you’ve had over a certain period of worker hours you’re interested in. You then normalize that rate over 200,000 worker hours because that’s the current standard. The more recordable incidents you have the higher your TRIR would be and vice versa.
Traditionally, if a company’s TRIR is better than the industry standard, it corresponds to better safety performance and is usually rewarded with incentives and performance bonuses. If it’s below the industry standard, then you’re essentially kicked out of the boardroom.
Safety professionals have strong feelings about this number and what it means. Sometimes safety systems take on progressive approaches to address serious injuries and fatalities but somehow a TRIR remain stubbornly higher and you may be at a loss over what to do. Even though the decisions that were made align with what progressive safety concepts are – moving toward a better culture, adopting more leading activities and relying less on lagging numbers – that TRIR number refuses to change.
Not the most accurate reflection of performance
TRIR is very important across the board because every organization uses this number and compares itself to others based on this number when it comes to safety.
But, all things considered, are we using TRIR appropriately?
There has been some rumbling among safety professionals that TRIR is not the most accurate reflection of the capacity or performance of a safety system that an organization has adopted.
We need to understand where it’s not working and how we should be using it, if at all.
A ridiculous example
Here is a somewhat ridiculous example. It’s possible, but the question to ask is, is it meaningful?
Company A had a recordable incident in the first 1,000 worker hours that they’re in business. This is unlikely, but it is possible.
At that point, their TRIR is 200 per 200,000 worker hours. The industry standard is 1.0. Two hundred is meaningless. We have no understanding of what 200 as a TRIR even means. This is so far off the scale of magnitude in horrible performance that numbers like 200, 250 and 300 are all the same at that point.
So you have to ask yourself, if this number isn’t meaningful then what are we doing with TRIR and what do we expect from this number?
What we need to ask next is do we expect another recordable incident in the next 1,000 worker hours? Are we expecting two? Or are we expecting none? What is that one incident in the first 1,000 worker hours of Company A telling us?
It’s not telling us much because we don’t have enough data.
Comparing more traditional examples
Let’s look at Company B, which has more traditional TRIR performance with seven recordable incidents over one million worker hours with a TRIR of 1.4 over 200,000 worker hours.
For Company C, we have 24 recordable incidents over six million worker hours. Company C’s data gives us a TRIR of 0.8.
On paper, Company C is better than Company B. However, we have a Company C that’s boasting a higher number of recordable incidents which is being masked by the higher number of worker hours.
Company B, on the other hand, has fewer recordable incidents with fewer worker hours.
The question to ask here is when are these ratios comparable? Because the worker hours aren’t equal and they never will be. Organizations have a different number of worker hours they engage in and a different number of recordable incidents.
So is Company C really better than Company B? According to traditional existing knowledge, the answer is yes. However, think about what we would expect from their performance in the future. Do we expect Company C to consistently stay at 0.8 and do better than Company B? Should we really select one over the other?
That’s the real question that needs to be asked when looking at these numbers.
TRIR doesn’t accurately predict future stats, fatalities
Research from the University of Colorado Boulder shows that it takes nine years of data to predict the TRIR of the tenth year, so trying to predict TRIR from year one to year two makes no sense. Beyond that, and up to year 10, trying to make predictions is statistically not reliable.
We also found that TRIR is no different than a random data set. You can put in any data set and give it the proper thresholds and it’ll generate any random number. It’s 98% random – almost like tossing a coin.
Bottom line: TRIR is not predictive of future TRIR unless you go out to nine years worth of data. And then beyond that ninth year, you’re really not comparing apples to apples since your safety system can change a lot over time.
TRIR also doesn’t accurately predict fatalities since, thankfully, fatal incidents are very rare. Because they’re so rare, they become statistical nightmares when it comes to trying to predict them.
Moving toward leading indicators, other metrics
Safety professionals have always had an inkling that TRIR wasn’t really working and that’s why we’ve moved more toward leading indicators, other metrics and a focus on safety culture.
According to Vector Solutions, a provider of Online Safety and Health Software, leading indicators play a vital role in preventing worker fatalities, injuries, and illnesses and allow you to take preventive action to address that failure or hazard before it turns into an incident. Using leading indicators can result in direct financial savings to the bottom line, including savings in repair costs, production costs, worker’s comp costs, and other legal and/or regulatory costs associated with incidents.
We’ve always known something wasn’t right in focusing too much on lagging indicators, but then the question becomes how do we move beyond TRIR?
A framework for judging safety metrics
How do we move forward? By finding more applicable metrics. To do that, we need a framework for judging the quality of any safety metric we have, including TRIR.
We need to ask, is the metric:
- predictive – can it tell us something about the future?
- valid – is what I’m measuring today the same thing I measured yesterday?
- objective – is it quantitative and not opinion-based?
- clear – how easy is it to understand?
- functional – can we do something about it?
- important – can we apply enough of our time and resources to dealing with it?
Not all metrics will touch perfectly on all of these points, but they should get close on most counts.
For example, if you apply this framework to TRIR, you’ll see that it is objective, clear and important but it isn’t at all predictive or valid when applied to a business unit.
(Adapted from a presentation by Siddharth Bhandari, Director of Construction Safety Research Alliance at the University of Colorado Boulder at the ASSP Safety 2022 Conference + Expo)
