Imperfect measure of hospital safety

The failure to include hospital-acquired infections or medication errors as a performance indicator limits the utility of the Canadian Institute for Health Information’s (CIHI) new hospital benchmarking tool, critics say.

Moreover, they add, the tool doesn’t provide adequate data, or adequate context, to interpret properly variations in its five patient safety indicators — in-hospital hip fracture among acute care inpatients age 65 and older, nursing-sensitive adverse events (such as urinary tract infections, pressure ulcers, in-hospital fractures and pneumonia) for all medical patients, nursing-sensitive adverse events for all surgical patients, obstetric trauma (fourth-degree lacerations or greater in severity) for instrument-assisted vaginal deliveries, and obstetric trauma for vaginal deliveries without instrument assistance (www.cihi.ca/CIHI-ext-portal/internet/en/documentfull/health+system+performance/indicators/performance/indicator_ent).

“Some hospitals are just unsafe and there’s no question about that. So the numbers are useful. But in the middle ground it may very well be for a lot of hospitals, these numbers are going to jump around regardless of what people do [because the tool only opens a narrow window onto the issue],” says Sholom Glouberman, president of the Patients’ Association of Canada.

The failure to include hospital-acquired infections and medication errors as indicators of hospital performance was a function of obstacles in data collection, according to CIHI, which developed the online tool (www.cmaj.ca/lookup/doi/10.1503/cmaj.109-4179).

But any accurate measure of patient safety within Canadian hospitals would have to include hospital-acquired infection rates, given that Canada has one of the worst hospital infection rates (11.6%) among developed countries, according to a 2011 World Health Organization report on the burden of endemic health care–associated infections (http://whqlibdoc.who.int/publications/2011/9789241501507_eng.pdf). By contrast, Germany’s rate is 3.6%, while that of France is 4.4%.

The failure to include hospital-acquired infections or medication errors as a performance indicator limits the utility of the Canadian Institute for Health Information’s new hospital benchmarking tool, critics say.

Image courtesy of ©2012 Thinkstock

In addition to hospital-acquired infection rates, Glouberman would like to see the tool report on avoidable admissions and patient experience.

It also would have made sense for CIHI to track medication errors in institutions and adverse drug reaction admissions, Linda Wilhelm, an independent member of the Best Medicines Coalition, a national alliance of advocacy groups that represents people living with chronic diseases, writes in an email. Such errors and reactions are a good place to start in weighing hospital performance and their inclusion would allow poor performers to “benefit from processes in other jurisdictions and institutions that demonstrated better safety,”

By current estimates, only 1%–10% of adverse drug reactions are reported in Canada, Wilhelm writes. “Medication safety has not been a priority for government, very little funding is put into this area despite the fact that medication errors and adverse drug reactions represent a significant cost to the health care system.”

CIHI hopes to eventually include such indicators within the database as it expands the project over time, says Jeanie Lacroix, manager of hospital reports for the agency.

But the relative lack of clinical detail within administrative data and coding irregularities continue to pose an obstacle to compiling comparable statistics on patient safety, Lacroix says. “It’s not the most straightforward thing [to track] because it happens at many different levels.”

Medication errors, for example, are not captured in administrative data, Lacroix notes. “We’re trying to hone in on the things we can collect with the data we have. … Culturally in hospitals there’s also sometimes still the fear you might get blamed or shamed if you report these things. I think we’re getting much better on that front but until everyone’s comfortable sharing those numbers it’s going to affect how the data settles.”

Lacroix also cautions that comparisons drawn from the data must be interpreted carefully. “We see variations but we can’t really explain them right now.” For example, medical patients in British Columbia and surgical patients in Newfoundland were more than twice as likely to experience adverse events related to nursing than similar patients in Prince Edward Island in fiscal 2009/10. But the extent to which that’s a reliable indicator of the relative safety of the hospitals in those provinces is unclear, she adds. “These are rare events so changes [even by one case] can result in a notable difference in a rate. Unlike other things with larger volumes, we can’t statistically say this is really a significant variation. We just don’t know.”

Others say the lack of context regarding patient safety indicators makes the data subject to misinterpretation and may lead some to draw the wrong conclusions.

For example, the tool indicates that hospitals in Ontario and British Columbia were among the worst performers for nursing-sensitive adverse events in 2009/10 but doesn’t reveal that the two provinces have the lowest number of nurses in terms of hours of care per capita, says Linda Haslam-Stroud, president of the Ontario Nurses’ Association. “The research is evident, although it isn’t in the CIHI report, that as you increase the number of patients you assign to a registered nurse, adding just one patient to my assignment increases morbidity and mortality rates 6%–7%.”

Patient safety, Haslam-Stroud adds, “starts from the top. What kind of cultures are the senior teams and management putting forward? What kind of values, working conditions and supports are they providing for [health] workers?”

Poor hospital performance with respect to patient safety may also be a function of access to care, rather than a reflection of quality of care, Glouberman notes. “The general story is Ma comes into hospital with chest pains, is put on a gurney for 18 hours and she has sensitive skin so she gets a bed sore. It might mean [the hospital doesn’t] have a proper emergency set up, but it doesn’t necessarily mean it’s an adverse nursing event.”