Hospital standardized mortality ratios

[The authors respond:]

The finding of higher rates of preventable deaths in hospitals with high mortality in the study by Dubois and colleagues1 applied only to the analysis of deaths from pneumonia, for which the physician reviewers exhibited very poor agreement (kappa = 0.11). Moreover, in citing Dubois and colleagues in our commentary,2 we did not presuppose that process problems constitute the gold standard for quality indicators. However, process change represents the major aspect of health care delivery under providers' control. If hospital standardized mortality ratios correlate poorly with the need for process changes (as in the study by Dubois and colleagues and a recent study from Ontario3), it remains unclear how hospital standardized mortality ratios can serve as a useful screen for quality problems.

Few would argue there are quality problems in the Canadian health care system. The Canadian Adverse Event Study found preventable events in every hospital studied.4 Ideally, all hospitals would accept these results as fact and undertake vigorous efforts to look for quality problems rather than wait for the results of their hospital standardized mortality ratios analysis. Given that this does not occur, one might argue for the use of a screening test, to engage hospitals.

However, as we outlined in our commentary, the hospital standardized mortality ratio has both low sensitivity and poor specificity for quality problems.2 This is not unheard of among screening tests. Despite terrible performance characteristics, the fecal occult blood test improves detection of colon cancer, presumably because the results of annual application of this test randomly scare sufficient numbers of patients into undergoing the test they should have agreed to undergo in the first place, namely colonoscopy.

Unfortunately, whereas colon cancer really does reside in the colon, most quality problems do not manifest themselves in the charts of deceased patients.5 Thus, rather than engaging hospitals in vigorous and effective detection of quality problems, promotion of hospital standardized mortality ratios focuses hospitals' attention on chart reviews of in-hospital deaths, which has all the inconvenience of colonoscopy but not comparable benefits.