Objective: To study the reliability and validity of concurrent review of hospital-bed utilization carried out by a trained nurse.
Design: Analysis of interrater reliability and validity of utilization review.
Setting: Tertiary care hospital associated with a university.
Patients: Eighty patients randomly selected from 203 patients admitted to the hospital.
Interventions: Appropriateness of days of stay in hospital was classified prospectively, on the basis of clinical judgement, by two nurses working independently, by a third nurse working with the Appropriateness Evaluation Protocol (AEP) and by a multidisciplinary review panel of nurses and physicians working retrospectively with the use of data gathered by the first nurse.
Main outcome measures: Agreement between different rates on the number of and reason for inappropriate admission days, total number of inappropriate days and of inappropriate days due to delayed discharge, to diagnostic procedures or to inefficient medical management.
Results: Agreement between the two nurses who used clinical judgement was substantial (kappa or the intraclass correlation coefficient [RI] 0.77 to 0.98 on the number of and reason for inappropriate admission days, on the total number of inappropriate hospital days and on days due to delayed discharge, diagnostic procedures or inefficient medical management. Agreement was moderate (RI 0.47) on the number of inappropriate day's stay awaiting surgery. Agreement was substantial (kappa or RI 0.69 to 0.94) between the two nurses who used clinical judgement and the panel, except on the total number of inappropriate days; however, for this variable, exclusion of one case increased the RI from 0.35 to 0.80. Agreement was substantial between the two nurses who used clinical judgement and the nurse who used the AEP on appropriateness of admission days and the number of inappropriate days. Agreement between the panel and the nurse who used the AEP on the number of inappropriate days rose from 0.36 to 0.88 when the one outlying case was excluded. Some admissions were classified as premature when the AEP was used, whereas other raters considered the admissions unnecessary. There was poor agreement between the nurse who used the AEP and the other raters on the number of inappropriate days' stay awaiting surgery or diagnostic tests.
Conclusions: Data collection and judgement of appropriateness of hospital stay by a trained nurse is feasible and reliable. A nurse working prospectively and a panel working retrospectively sometimes disagree. The AEP provides a similar estimate of the number of inappropriate days but may be insensitive to patient factors that influence the timing of admission.