Healthcare News & Insights

Reason why safety scores may not be completely accurate

Hospitals are often judged based on their performance with safety rankings, most notably those from the Leapfrog Group. However, these “safety report cards” may have issues with accuracy for several reasons – and they could actually penalize hospitals for their honesty. 

A news release from the University of Michigan describes research published in the Medical Care journal that examines the Leapfrog Group’s Hospital Safety Grade system.

The biggest issue with the rankings, according to researchers, comes from their reliance on hospitals’ accuracy when self-reporting Safe Practices measures on the voluntary Leapfrog Hospital Survey.

Perceptions of safety

Many hospitals tended to give themselves near-perfect scores when assessing these practices, which include measures like clinicians’ hand hygiene and communication. And that tends to give them higher rankings on Leapfrog’s list.

However, when reviewing self-reported scores and comparing them to independent, third-party measurements of hospitals’ performance in areas like readmissions and hospital-acquired infections, researchers found that self-reported rankings had little to no impact on hospitals’ actual performance with patient safety.

Plus, these self-reported scores can actually make high-ranking hospitals look as if they’re low performers. The individual performance of each hospital that self-reports data to the Leapfrog Group is compared against an average of the data from all other hospitals. This comparison factors into a facility’s overall score.

Hospitals that don’t rank themselves highly with certain Safe Practices measure up more poorly against the average, which may lower their overall score – even if they perform well in other areas.

To show just how much these self-reported scores affect a hospital’s end score overall, researchers examined the rankings of nonparticipating hospitals that didn’t report Safe Practices scores. Although Safe Practices scores count for 25% of a hospital’s ranking if it self-reports data, they aren’t counted at all when calculating the ranking for nonparticipating hospitals.

With that in mind, researchers calculated how much a nonparticipating hospital’s score would change if it had reported low, average or perfect performance with Safe Practices measures, looking at their infection rates for comparison.

Result: Hospitals that did well with preventing infections would still have lower safety rankings from the Leapfrog Group if they reported anything less than perfect performance with Safe Practices measures.

In fact, researchers suggested that hospitals who had low or average scores on their self-reported surveys may be better off just not completing the survey at all. Scores would be higher if Leapfrog didn’t factor in their performance.

The Leapfrog Group is supposed to change its formula soon to put less focus on self-reported information from these surveys, which may possibly give more hospitals better safety scores, but that’s still months down the line.

In the meantime, researchers said, this is all the more reason to stop judging hospital performance based on surveys and safety rankings, and to start based on publicly available information about how well a facility is doing with preventing infections and other complications.

Subscribe Today

Get the latest and greatest healthcare news and insights delivered to your inbox.