Validity of utilization review tools

Norman Kalant and colleagues conclude that utilization review tools are not valid to assess appropriateness of setting.1 We argue that this conclusion is not supported for several reasons. First, and most importantly, acute care review tools assume that subacute care and acute care are separate, discrete levels. With only minor exceptions, in Canada subacute care is normally and appropriately delivered within the acute care setting. One report referred to by the authors specifically makes this point in terms of the structure of the Ontario health system.2 This oversight alone is likely to account for a significant proportion of the mismatch between the review tools for acute care (without including subacute care) and the opinion of a panel of experts.

The authors base their conclusions on a simple kappa statistic. However, they have not adjusted for the nonindependent nature of the days of stay of the 75 patients reviewed. Our work has demonstrated a 30% correlation between the appropriateness of 1 day of stay and appropriateness of the subsequent day (unpublished data). Nonindependence of observations may amplify disagreement as measured by the kappa statistic.

Finally, the authors claim that no previous studies have validated these tools by comparison with implicit review by a panel of physicians. In fact 2 published studies used physician panels to demonstrate validity.3^,4