Abstract
Background: Readmissions to hospital are common, costly and often preventable. An easy-to-use index to quantify the risk of readmission or death after discharge from hospital would help clinicians identify patients who might benefit from more intensive post-discharge care. We sought to derive and validate an index to predict the risk of death or unplanned readmission within 30 days after discharge from hospital to the community.
Methods: In a prospective cohort study, 48 patient-level and admission-level variables were collected for 4812 medical and surgical patients who were discharged to the community from 11 hospitals in Ontario. We used a split-sample design to derive and validate an index to predict the risk of death or nonelective readmission within 30 days after discharge. This index was externally validated using administrative data in a random selection of 1 000 000 Ontarians discharged from hospital between 2004 and 2008.
Results: Of the 4812 participating patients, 385 (8.0%) died or were readmitted on an unplanned basis within 30 days after discharge. Variables independently associated with this outcome (from which we derived the nmemonic “LACE”) included length of stay (“L”); acuity of the admission (“A”); comorbidity of the patient (measured with the Charlson comorbidity index score) (“C”); and emergency department use (measured as the number of visits in the six months before admission) (“E”). Scores using the LACE index ranged from 0 (2.0% expected risk of death or urgent readmission within 30 days) to 19 (43.7% expected risk). The LACE index was discriminative (C statistic 0.684) and very accurate (Hosmer–Lemeshow goodness-of-fit statistic 14.1, p = 0.59) at predicting outcome risk.
Interpretation: The LACE index can be used to quantify risk of death or unplanned readmission within 30 days after discharge from hospital. This index can be used with both primary and administrative data. Further research is required to determine whether such quantification changes patient care or outcomes.
See related commentary by Goldfield, page 538
Readmission to hospital and death are adverse patient outcomes that are serious, common and costly. 1,2 Several studies suggest that focused care after discharge can improve post-discharge outcomes. 3–7 Being able to accurately predict the risk of poor outcomes after hospital discharge would allow health care workers to focus post-discharge interventions on patients who are at highest risk of poor post-discharge outcomes. Further, policy-makers have expressed interest in either penalizing hospitals with relatively high rates of readmission or rewarding hospitals with relatively low expected rates. 8 To implement this approach, a validated method of standardizing readmission rates is needed. 9
Two validated models for predicting risk of readmission after hospital discharge have been published. 10,11 However, these models are impractical to clinicians. Both require area-level information (e.g., neighbourhood socio-economic status and community-specific rates of admission) that is not readily available. Getting this information requires access to detailed tables, thereby making the model impractical. Second, both models are so complex that risk estimates cannot be attained from them without the aid of special software. Although these models have been used by health-system planners in the United Kingdom, we are unaware of any clinicians who use them when preparing patients for hospital discharge.
Our primary objective was to derive and validate a clinically useful index to quantify the risk of early death or unplanned readmission among patients discharged from hospital to the community.
Methods
Study design
We performed a secondary analysis of a multicentre prospective cohort study conducted between October 2002 and July 2006. The study involved patients discharged to the community from the medical or surgical services of 11 hospitals (6 university-affiliated, 5 community) in five cities in Ontario after an elective or emergent hospital admission. To be eligible for inclusion, patients had to be adults, provide informed consent, have a telephone (to participate in follow-up telephone interviews), and be cognitively intact (to ensure validity of the consent process and accuracy of information given in the interviews). We recruited patients of medical and surgical services because such patients comprise most discharges from hospitals.
Given that the process of health care provided in nursing homes differs from processes of care for patients in the community (i.e., nursing home residents frequently receive care from an onsite physician), we restricted eligibility to patients who were not residents of nursing homes.
Before discharge from hospital, patients were interviewed by study personnel to identify their baseline functional status, living conditions and chronic medical conditions. Chronic medical conditions were confirmed by a review of the patient’s chart and hospital discharge summary, when available. The chart and discharge summary were also used to identify diagnoses made while in hospital and medications given at discharge. All medications given at discharge were compared with those documented on the admission note to determine which discharge medications had been started in hospital.
To determine whether patients had had an unplanned readmission to hospital or had died within 30 days of discharge, we contacted either patients or their principal contacts (identified by each patient at recruitment) one month after discharge. We combined unplanned readmissions with deaths to avoid bias caused by censoring deaths when hospital readmission alone is examined. 12 We classified readmissions as unplanned if they had not been arranged or planned when the patient was originally discharged from hospital.
We chose a 30-day time frame for our primary outcome to increase the likelihood that poor outcomes would be related to the index admission or discharge process and would be more likely to be remediable. When analyzing this outcome by its components, we classified patients who were urgently readmitted within 30 days as having died if they subsequently died within 30 days of discharge from hospital.
Our study was approved by the Ottawa Hospital Research Ethics Board and by each participating site.
Index derivation and internal validation
We randomly selected half of the participants for index derivation and used the other half for internal validation. We identified all patient-level and admission-level variables among the data set that we thought might influence outcomes. We used multivariable logistic regression to measure the independent association of these factors with early death or unplanned readmission to hospital. Patient-level variables and admission-level variables entered into the model are listed in Table 1. We also offered the presence or absence of most common diagnoses and procedures (Appendix 1, available at www.cmaj.ca/cgi/content/full/cmaj.091117/DC1).
Table 1: Characteristics of 4812 participants discharged from hospital to the community, by outcome within 30 days after discharge
We used fractional polynomial functions to determine the best linear or nonlinear form for continuous variables. 15–17 Backward stepping with an α-error criterion of 0.01 was used to include only significant variables in the final multivariable logistic model. We used an inclusion criterion of 1% to ensure model stability. To account for clustering of patients within hospitals, we used generalized estimating equation methods. We found no changes in the parameter estimates for all model variables, and all variables remained significant at the 5% level.
We then used the methods described by Sullivan and colleagues 18 to modify the final logistic model into a risk index. The number of points assigned to each significant covariate equaled its regression coefficient divided by the parameter estimate in the model with the smallest absolute value rounded to the nearest whole number. We then calculated each participant’s final score by summing up his or her points. The expected probability of early death or unplanned readmission associated with each score was the inverse of 1 + e–(intercept+b*total score), where b was the value of the coefficient in the regression model with the smallest absolute value.
External validation
Since the components of the final model were available from administrative data, the index was externally validated using three population-based administrative databases that capture data on all Ontarians. The Discharge Abstract Database records all hospital admissions. The National Ambulatory Care Reporting System records all emergency department visits, and the Registered Patient Database records all dates of death.
We used the Discharge Abstract Database to randomly select 1 000 000 (of 1 916 398) adult medical or surgical patients out of discharged to the community from Ontario hospitals between April 2004 and January 2008. This period was used to ensure that six months of preadmission data existed in the National Ambulatory Care Reporting System for all participants. The urgency and length of stay of each hospital admission were noted.
We calculated each patient’s Charlson comorbidity index score using the International Classification of Disease (ICD) codes cited by Quan and colleagues. 19 We linked to the National Ambulatory Care Reporting System to measure the number of visits to an emergency department by each patient in the six months before admission. We determined patients’ status at 30 days post-discharge by linking to the Registered Patient Database for data related to deaths and to the Discharge Abstract Database for data related to unplanned urgent readmissions.
Assessment of risk score
We used a C statistic with 95% confidence intervals (CIs) 20 to measure the ability of the index to discriminate between patients who died or had an unplanned readmission within 30 days of discharge and those who did not. The C statistic expresses the proportion of times that the case in each case–noncase pair has a higher model-based predicted risk of the outcome. We measured the calibration of the score by comparing the observed and expected numbers of patients with the outcome for each score. We deemed the observed and expected death or urgent readmission rates to be similar if the 95% CI around the observed rate included the expected rate. We calculated 95% CIs for observed rate of death or urgent readmission rates using exact methods. 21 We summarized overall calibration using a Hosmer–Lemeshow goodness-of-fit test. 22
Results
Between October 2002 and July 2006, we enrolled 5035 patients from 11 hospitals. We determined outcome status at 30 days for 4812 patients (95.6%). Of the remaining patients, 124 (2.5%) refused participation when contacted for follow-up, 83 (1.6%) were lost to follow-up, and 16 (0.3%) were removed from the study because they were admitted to a nursing home during the first month after discharge from hospital.
The study cohort is described in Table 1. Participants were middle-aged, and almost 95% were independent with regard to activities of daily living. Most participants were free of serious comorbidities, with more than 75% having a Charlson comorbidity index score of zero. 13 Most admissions were emergent (58.1%), and almost half (44.9%) were to a medical service. The most common reasons for hospital admission included acute coronary syndromes, cancer diagnosis and complications, and heart failure (Appendix 1). Coronary artery bypass grafting and arthroplasty were the most common procedures. Patients in the derivation (n = 2393) and validation (n = 2419) cohorts were similar.
During the first 30 days after discharge, 385 (8.0%) patients died or were urgently readmitted (death 36 [9.4% of outcomes], unplanned readmission 349 [90.6% of outcomes]). Patients with one of the primary outcomes had more emergency department visits before admission and were more likely to be admitted emergently and for longer durations than patients who did not die (Table 1). Most other patient-related and admission-related variables appeared to have little influence on risk of early death or unplanned readmission.
Index derivation and internal validation
Only four variables were independently associated with death or readmission within 30 days after discharge (Table 2). These variables were length of stay (“L;” odds ratio [OR] 1.47, 95% CI 1.25–1.73) acuity of the admission (“A;” odds ratio [OR] 1.84, 95% CI 1.29–2.63), patient comorbidity (as measured using the total Charlson comorbidity index score) (“C;” odds ratio [OR] 1.21, 95% CI 1.10–1.33), and emergency department use (measured as the number of visits in the previous six months) (“E;” odds ratio [OR] 1.56, 95% CI 1.27–1.92). Length of stay was modelled as a logarithm and the number of emergency department visits was modelled as a square root term. We found no significant interactions between these or other variables. The final logistic model was moderately discriminative (C statistic 0.700) and was well calibrated (Hosmer–Lemeshow goodness of fit statistic 6.99, 8 degrees of freedom, p = 0.54). None of the other variables listed in Table 1 met our criteria for inclusion in the model.
Table 2: Final logistic regression model for risk of death or unplanned readmission within 30 days after discharge (derivation group only, n = 2393)
We modified this logistic model into an index to predict early death or unplanned readmission (Table 3). To facilitate recall of the components of the index, we titled the index using a simple mnemonic. The LACE index had a potential score ranging from 0 to 19. The total LACE score in the study population had a normal distribution that was slightly skewed to the right (Figure 1).
Table 3: LACE index for the quantification of risk of death or unplanned readmission within 30 days after discharge
Figure 1: Calibration curve for the LACE index, based on data representing patients in the derivation and internal validation groups. Note: bars = number of patients with the same LACE score; black line = expected risk of death or unplanned readmission within 30 days after discharge; grey line = observed risk (error bars = 95% confidence intervals).
The LACE index had moderate discrimination for early death or readmission. The C statistic (95% CI) in the derivation was 0.7114 (0.6736–0.7491). In the validation, it was 0.6935 (0.6548–07321), and in the entire cohort, it was 0.7025 (0.6755–0.7295).
The expected probability of death or readmission within 30 days of discharge for each point ranged from 2.0% for a LACE score of 0 to 43.7% for a LACE score of 19 (Table 4). The expected probability of early death or unplanned readmission was within the 95% CIs of the observed rates for all LACE scores in both the derivation and validation cohorts (Table 4) as well as the entire cohort (Figure 1). The Hosmer–Lemeshow statistic in the derivation was 18.7 (p = 0.42). In the validation, it was 14.1 (p = 0.59), and in the entire cohort, it was 21.2 (p = 0.27) (Table 4).
Table 4: Expected and observed probability of death or unplanned readmission within 30 days after discharge, by LACE score
The LACE score was strongly associated with each outcome individually. A 1-point increase in the LACE score increased the odds of unplanned readmission by 18% (odds ratio 1.18, 95% CI 1.14–1.21). The LACE index in the entire cohort was moderately discriminative for 30-day unplanned readmission (C statistic 0.679, 95% CI 0.650–0.708) and well calibrated (Hosmer–Lemeshow statistic 11.5, p = 0.18). A one-point increase in the LACE score increased the odds of early death by 29% (odds ratio 1.29, 95% CI 1.20–1.38). The LACE index was very discriminative for early death (C statistic 0.793, 95% CI 0.733–0.854) and well calibrated (Hosmer–Lemeshow statistic 4.7, p = 0.79).
External validation
The external validation group contained 1 000 000 randomly selected patients (mean age 59.1, standard deviation [SD] 18.4 years; 48.4% female). Patients had a mean length of stay of 5.1 days (SD 7.7), a mean Charlson comorbidity index score of 0.5 (SD 1.2), and a mean number of emergency department visits of 0.4 (SD 7.9), with 67.6% of the index admissions emergent. Patients had a mean LACE score of 6.0 (SD 3.1) and 7.8% of patients died (1.1%) or were urgently readmitted (7.3%) within 30 days of discharge. Discrimination of the LACE index was the same in this patient group (C statistic 0.684, 95% CI 0.679–0.691). The observed rate of early death or urgent readmission slightly exceeded the expected rates at most LACE scores (Figure 2). However, the median absolute difference between expected and observed rates was small, at 1.6% (range 0.04%–6.6%).
Figure 2: External validation of the LACE index, as represented by its accuracy for 1 000 000 randomly selected patients discharged from hospital in Ontario between 2004 and 2008. Note: bars = number of patients with the same LACE score; black line = expected risk of death or unplanned readmission within 30 days after discharge; grey line = observed risk (error bars = 95% confidence intervals).
Interpretation
We have derived and validated an easy-to-use index that is moderately discriminative and very accurate for predicting the risk of early death or unplanned readmission after discharge from hospital to the community. Further research is required to determine whether such quantification changes patient care or outcomes.
We found its simplicity very notable. Although we derived the LACE index in a large cohort of patients using almost 50 factors — each of which could reasonably influence the risk of post-discharge outcomes — we found that four simple factors explained much of the variation in risk of early death or unplanned readmission after discharge from hospital. The LACE index therefore joins other indexes in which seemingly complex outcomes are predicted with a few simple factors. 23
The LACE index has several strengths to support its use. 24 The outcome predicted by the index is important, clinically relevant and reliably measured. Determination of this outcome for each patient was independent of the LACE score. Each component of the LACE index is readily and reliably determined. The methods we used to derive the LACE index were both valid and transparent. The discrimination of the LACE index was better than that of the widely used Framing-ham score in many populations, 25–27 which suggests that the LACE index will be useful when applied at the individual patient level. The calibration of the LACE index was excellent, which suggests that it will also be useful when applied by policy-makers. Finally, the LACE index is easier to use than previous models, because the latter require variables —such as community admission rates 28 or area-level socio-economic measures 29 — that are usually unavailable to clinicians.
Limitations
Three main limitations about the LACE index should be noted. First, the index cannot be used reliably in patient populations that were not involved in its derivation. Second, further work is required to identify additional factors that may increase the discrimination or accuracy of the index. Third, clinicians will find it difficult to commit to memory the point system and its expected risks. Therefore, use of the LACE index will usually require a computational aid. Until the LACE index is externally validated with primary data, we recommend that it be used for outcomes research and quality assurance rather than in decision-making for individual patients.
Conclusion
Notwithstanding its limitations, we believe that the LACE index can be used by researchers and administrators to predict the risk of early death or unplanned readmission of cognitively intact medical or surgical patients after discharge from hospital to the community. Further research is required to determine whether quantifying the risk of poor outcomes after discharge actually changes patient care or outcomes.
Footnotes
-
Previously published at www.cmaj.ca
This article has been peer reviewed.
Competing interests: None declared.
Contributors: All of the authors were involved in the conception and design of the study, the acquisition of data, the analysis or interpretation of data, the drafting of the manuscript and the critical revision of the manuscript for important intellectual content. All of them approved the final version submitted for publication. Dr. van Walraven had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Funding: This study was funded by the Canadian Institutes of Health Research, the Physicians’ Services Incorporated Foundation and the Department of Medicine, University of Ottawa.
REFERENCES
In this issue
Article tools
Jump to section
Related Articles
Cited By...
- Targeted Clinical Interventions for Reducing Pediatric Readmissions
- Social Vulnerability Indices as a Risk Factor for Heart Failure Readmissions
- Evaluation of an intervention targeted with predictive analytics to prevent readmissions in an integrated health system: observational study
- Published models that predict hospital readmission: a critical appraisal
- Exploring instruments used to evaluate potentially inappropriate medication use in hospitalised elderly patients in Kosovo
- Accurate and reproducible prediction of ICU readmissions
- Combining structured and unstructured data for predictive models: a deep learning approach
- Improving Hospital Readmission Prediction using Individualized Utility Analysis
- Reader Response: Neuro R2 Score: Predicting High-Risk Neurologic Readmissions Within 30 Days
- Heart Failure with Preserved Ejection Fraction and 30-Day Readmission
- HOSPITAL Score, LACE Index and LACE+ Index as predictors of 30-day readmission in patients with heart failure
- Improving transitions from acute care to home among complex older adults using the LACE Index and care coordination
- Neuro R2 score: Predicting high-risk neurologic readmissions within 30 days
- Predicting and preventing hospital readmission for exacerbations of COPD
- Ability of the LACE index to predict 30-day hospital readmissions in patients with community-acquired pneumonia
- Medication-related harm in older adults following hospital discharge: development and validation of a prediction tool
- Vital Sign Abnormalities on Discharge Do Not Predict 30-Day Readmission
- Predicting risk of unplanned hospital readmission in survivors of critical illness: a population-level cohort study
- Predictors of 30-day hospital readmission: The direct comparison of number of discharge medications to the HOSPITAL score and LACE index
- Risk scores identifying elderly inpatients at risk of 30-day unplanned readmission and accident and emergency department visit: a systematic review
- Protocol of a multimethod descriptive study: adapting hospital-to-home transitional care interventions to the rural healthcare context in Ontario, Canada
- Disparities in health outcomes among seniors without a family physician in the North West Local Health Integration Network: a retrospective cohort study
- Impact of oral anticoagulants on 30-day readmission: a study from a single academic centre
- Death and readmissions after hospital discharge during the December holiday period: cohort study
- Factors associated with attendance at primary care appointments after discharge from hospital: a retrospective cohort study
- Development of a risk predictive scoring system to identify patients at risk of representation to emergency department: a retrospective population-based analysis in Australia
- The Cleveland Clinic Pharmacy Population Health Management Summit
- A Novel Model for Enhanced Prediction and Understanding of Unplanned 30-Day Pediatric Readmission
- Determinants of patient choice for hospital readmission after township hospitalisation: a population-based retrospective study in China
- Necessity for a pathway for "high-alert" patients
- The Effect of Malnutrition on the Risk of Unplanned 7-Day Readmission in Pediatrics
- Emergency department presentation and readmission after index psychiatric admission: a data linkage study
- Predictive risk modelling under different data access scenarios: who is identified as high risk and for how long?
- Development and validation of a transitions-of-care pharmacist tool to predict potentially avoidable 30-day readmissions
- Acute Myocardial Infarction Readmission Risk Prediction Models: A Systematic Review of Model Performance
- Mortality of hospitalised internal medicine patients bedspaced to non-internal medicine inpatient units: retrospective cohort study
- Investigating the effect of sociodemographic factors on 30-day hospital readmission among medical patients in Toronto, Canada: a prospective cohort study
- Patients and providers perceptions of the preventability of hospital readmission: a prospective, observational study in four European countries
- National Estimates of 30-Day Unplanned Readmissions of Patients on Maintenance Hemodialysis
- Preventing hospital readmissions: the importance of considering 'impactibility, not just predicted risk
- Caring about what happens to people with COPD
- The PEARL score predicts 90-day readmission or death after hospitalisation for acute exacerbation of COPD
- Evaluating the predictive strength of the LACE index in identifying patients at high risk of hospital readmission following an inpatient episode: a retrospective cohort study
- Burden of Readmissions Among Patients With Critical Limb Ischemia
- Utility of models to predict 28-day or 30-day unplanned hospital readmissions: an updated systematic review
- Admission Data Predict High Hospital Readmission Risk
- Rehospitalizations and Emergency Department Visits after Hospital Discharge in Patients Receiving Maintenance Hemodialysis
- Physician experience and outcomes among patients admitted to general internal medicine teaching wards
- Association between frailty and 30-day outcomes after discharge from hospital
- A tool for prediction of risk of rehospitalisation and mortality in the hospitalised elderly: secondary analysis of clinical trial data
- Safely and effectively reducing inpatient length of stay: a controlled study of the General Internal Medicine Care Transformation Initiative
- The effect of a collaborative pharmacist-hospital care transition program on the likelihood of 30-day readmission
- Causes and patterns of readmissions in patients with common comorbidities: retrospective cohort study
- Do Outcomes for Patients With Heart Failure Vary by Emergency Department Volume?
- Postdischarge Outcomes in Heart Failure Are Better for Teaching Hospitals and Weekday Discharges
- Postdischarge Environment Following Heart Failure Hospitalization: Expanding the View of Hospital Readmission
- How Health Systems Could Avert 'Triple Fail' Events That Are Harmful, Are Costly, And Result In Poor Patient Satisfaction
- Home-care nurses' perceptions of unmet information needs and communication difficulties of older patients in the immediate post-hospital discharge period
- Transitional interventions to reduce early psychiatric readmissions in adults: systematic review
- Development of a predictive model to identify inpatients at risk of re-admission within 30 days of discharge (PARR-30)
- Causes and patterns of readmissions in patients with common comorbidities: retrospective cohort study
- Predictors of Thirty-Day Readmission Among Hospitalized Family Medicine Patients
- Patterns and Costs of Health Care Use of Children With Medical Complexity
- An Index to Predict Unplanned Hospital Readmission
More in this TOC Section
Similar Articles
