Abstract
BACKGROUND: Prognostication tools that report personalized mortality risk and survival could improve discussions about end-of-life and advance care planning. We sought to develop and validate a mortality risk model for older adults with diverse care needs in home care using self-reportable information — the Risk Evaluation for Support: Predictions for Elder-Life in the Community Tool (RESPECT).
METHODS: Using a derivation cohort that comprised adults living in Ontario, Canada, aged 50 years and older with at least 1 Resident Assessment Instrument for Home Care (RAI-HC) record between Jan. 1, 2007, and Dec. 31, 2012, we developed a mortality risk model. The primary outcome was mortality 6 months after a RAI-HC assessment. We used proportional hazards regression with robust standard errors to account for clustering by the individual. We validated this algorithm for a second cohort of users of home care who were assessed between Jan. 1 and Dec. 31, 2013. We used Kaplan–Meier survival curves to estimate the observed risk of death at 6 months for assessment of calibration and median survival. We constructed 61 risk groups based on incremental increases in the estimated median survival of about 3 weeks among adults at high risk and 3 months among adults at lower risk.
RESULTS: The derivation and validation cohorts included 435 009 and 139 388 adults, respectively. We identified a total of 122 823 deaths within 6 months of a RAI-HC assessment in the derivation cohort. The mean predicted 6-month mortality risk was 10.8% (95% confidence interval [CI] 10.7%–10.8%) and ranged from 1.54% (95% CI 1.53%–1.54%) in the lowest to 98.1% (95% CI 98.1%–98.2%) in the highest risk group. Estimated median survival spanned from 28 days (11 to 84 d at the 25th and 75th percentiles) in the highest risk group to over 8 years (1925 to 3420 d) in the lowest risk group. The algorithm had a c-statistic of 0.753 (95% CI 0.750–0.756) in our validation cohort.
INTERPRETATION: The RESPECT mortality risk prediction tool that makes use of readily available information can improve the identification of palliative and end-of-life care needs in a diverse older adult population receiving home care.
Most people in high-income countries die of causes with progressive, predictable trajectories of decline.1–4 Since 2000, the 3 leading causes of death in Canada — accounting for 55% of all deaths — have been cancer, heart disease and stroke.1 Other leading causes of death, such as dementia and chronic lower respiratory diseases, also share signs and symptoms of senescence that are common across chronic diseases, including deterioration of physical and cognitive function, as well as an increased need for assistance.
Despite the predictable nature of most deaths, many Canadian residents who are at the end of life do not receive adequate home-based supports.5 In Ontario — the largest province in Canada with more than 14 million residents and the setting of this study — only 40% of decedents receive formal home care, and less than 20% receive a physician home visit in their last year of life.6,7 Even among those who had received palliative and end-of-life care, the start of service was often too close to death and failed to have a positive impact on the quality of life in those last months.8 The lack of available and accurate prognostic information is a key challenge. There are few existing tools that can be used to inform palliative care planning for the general population of older adults who live in the community and in people without cancer.9 Other barriers to accurate prognostic estimates include clinicians’ reluctance or lack of time and existing prognostication tools’ reliance on complex or specialized inputs, such as laboratory data and previous health care use. As a result, many older and frail adults do not receive timely palliative care and do not have an advance care plan.6,10–13
Our primary objective was to develop and validate a model for predicting mortality risk among the general population of community-dwelling adults with and without cancer that spans an actionable period for end-of-life planning (5 yr to imminent death). The variables included in our prognostication model — the Risk Evaluation for Support: Predictions for Elder-life in the Community Tool (RESPECT) — were prespecified to include exposures that could be self-reported by patients and their caregivers, including family members.
Methods
We derived and validated RESPECT using population-based home care data housed at ICES in Ontario. The reporting of our approach and findings adhere to the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement.14 A detailed protocol prespecifying our approach was published elsewhere.15
Study population
The study population included people who were 50 years of age or older, eligible for government-funded long-term home care in the community and had received at least 1 assessment using the Resident Assessment Instrument for Home Care (RAI-HC) between Jan. 1, 2007, and Dec. 31, 2013. The derivation cohort included those who used home care between Jan. 1, 2007, and Dec. 31, 2012, and the temporal validation cohort received care and were assessed between Jan. 1 and Dec. 31, 2013. The RAI-HC is a comprehensive, multidimensional instrument for clinical assessment that contains nearly 400 data elements that capture the home care client’s sociodemographic profile, cognitive and functional capacities, chronic diseases and comorbidities, and signs of health instability, as well as recent use of health care. It is primarily used for care planning within the home care setting. However, it also contains quality indicators and outcome measures that could be used to evaluate the impact of services provided. In Ontario, home care services, such as nursing, personal support and rehabilitative or restorative therapy, are publicly funded and provided to people to improve or manage their health based on assessed need. The RAI-HC is used by home care coordinators to evaluate the level of care need in those who are expected to require at least 60 days of uninterrupted service (also known as “long-stay clients”). Evaluations using the RAI-HC are performed at initial consideration for home care and completed at least once every 6 months for those receiving home care over an extended period, or when substantial changes in the client’s situation have been observed.
The primary outcome of our prediction model was death within 6 months of a RAI-HC assessment. Death was ascertained from the Registered Persons Database, a registry of health card numbers that have been issued under the Ontario Health Insurance Plan to all eligible residents of Ontario.
The selection of predictors included in our model was informed by clinical experience and our review of existing mortality prediction models for a population of older, community-dwelling adults. We also considered variables included in existing frailty indices.9 We considered risk factors related to physical functioning (e.g., difficulties with activities of daily living [ADL], inability to independently carry out instrumental activities of daily living [IADL] and reduced mobility), cognitive impairment (e.g., memory decline and psychosis), sociodemographic factors (e.g., level of education) and biological diseases (e.g., diabetes, heart disease and cancer). We also included self-reported measures of recent use of health care (i.e., number of hospital admissions or visits to the emergency department in the last 90 d), the prescription and receipt of life-sustaining therapies (e.g., dialysis and ventilation), and symptoms of reduced health and physiologic reserve (e.g., weight loss, edema and vomiting). The latter variables were selected to capture the acute symptomatology of people in the terminal period of life. We also incorporated cohort characteristics (e.g., year of the RAI-HC assessment, and the type of and reason for assessment) that may account for remaining heterogeneity in the estimated risks. No stepwise variable selection procedure was used.
Statistical analysis
We fit a Cox proportional hazards regression to predict death within 6 months of a RAI-HC assessment. We included all assessments for an individual user of home care that occurred within our model development timeframe (Jan. 1, 2007, to Dec. 31, 2012). We followed each assessment record until death or censored it at 6 months after the assessment date. We accounted for within-subject correlations using a robust sandwich variance estimator because of the possibility of home care users having multiple assessment records within this timeframe.16
Given our large data set and number of observed events, there was little concern about the sample size required for the number of predictors and degrees of freedom included in our final model. We modelled age using restricted cubic splines with 5 knots. Restricted cubic splines are piecewise cubic functions that are smooth at the knots and restricted to be linear in the tails. We placed the knots at fixed quantiles (i.e., the 5th, 27.5th, 50th, 72.5th and 95th percentiles) of the distribution of age. All other predictors, with the exception for age, were specified as categorical variables.
In the interest of having a parsimonious model, we removed predictors that could have been collinear with other predictors in the model (i.e., hip fracture and peripheral vascular disease), minimally contributed to the model’s predictive performance using a step-down method17 based on contribution to model R2 (i.e., marital status and diabetes) or lacked precision in measurement (i.e., any psychiatric diagnosis and daily pain). The relative contribution of these predictors to the initial model are presented in Appendix 1, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200022/tab-related-content. Our final derivation model included 28 predictors with 65 degrees of freedom.
Predictive performance was assessed in the validation cohort using predictive accuracy measures, including discrimination (e.g., C statistic) and calibration.18 To assess model calibration, we estimated the predicted risk at 6 months after a RAI-HC assessment by applying the coefficients from the derivation model to the validation cohort (Appendix 2, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200022/tab-related-content). For the year of assessment, we used the model coefficient from the final year of observation (i.e., 2012). We used Kaplan–Meier survival curves to obtain observed risks. We evaluated the overall model calibration, as well as calibration across all predictors included in RESPECT.
We combined the derivation and validation cohorts to derive the final estimates (Appendix 2) for algorithm implementation as a web-based tool. Regression estimates from the combined model did not differ noticeably from the derivation model.
Using the full study cohort (comprising RAI-HC assessments performed between Jan. 1, 2007, and Dec. 31, 2013), we constructed 61 risk groups (Appendix 3, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200022/tab-related-content) using median survival estimates derived from Kaplan–Meier survival curves. This serves as a scoring system for grouping people into clusters of patients with similar prognosis. Unlike the timeframe used in model development (i.e., 6 months from each RAI-HC assessment conducted between Jan. 1, 2007, and Dec. 31, 2012), we followed each RAI-HC assessment performed in our full cohort until death or the most recent data available at the time of this study (i.e., Dec. 31, 2018). This represents a maximum follow-up of 11 years for RAI-HC assessments performed in January 2007. The 61 risk groups represent incremental increases in median survival of about 3 weeks among higher-risk groups and between 2 and 3 months in lower-risk groups (Figure 1). Sample sizes for each risk group and their median survival are presented in Appendix 4, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200022/tab-related-content. These cut-offs were guided by expert consensus among clinical investigators in this study. For example, the highest risk groups (i.e., bins 1–5) comprised patients whose median survival was about 20 days (e.g., between patients in bin 1 and bin 2), 19, 14 and 12 days apart, respectively. Incremental changes between the risk bins were more considerable from bin 7 onwards (15–327 d), because the median survival in these risk groups was greater than 6 months.
Mean predicted 6-month mortality across risk groups. Note: The horizontal bars are the upper and lower bounds of the 95% confidence intervals.
We implemented the final combined model as a web-based risk communication tool hosted on a knowledge translation platform developed by our research team (available at www.projectbiglife.ca/elder-life-calculator). Screenshots of the web-based tool are provided in Appendix 5, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200022/tab-related-content.
Ethics approval
ICES is a prescribed entity under section 45 of Ontario’s Personal Health Information Protection Act. The use of data in this project was authorized under section 45 of the Act, which does not require review by a Research Ethics Board.
Results
Our derivation cohort comprised 435 009 community-dwelling older adults who used home care between Jan. 1, 2007, and Dec. 31, 2012 (Appendix 6, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200022/tab-related-content). There were 1 160 429 RAI-HC assessments performed for these adults during this period (comprising 4 122 086 person-years [PYs] of follow-up). The number of RAI-HC assessments per adult ranged from 1 to 25, with a median of 2 (interquartile range [IQR] 1–3). Of these, 103 681 adults (23.8% of the derivation cohort) died within 6 months of a RAI-HC assessment.
Our temporal validation cohort comprised 195 951 RAI-HC assessments that were completed between Jan. 1 and Dec. 31, 2013, and represented 139 388 home care users and 371 567 PYs of follow-up. The number of assessments per adult ranged between 1 and 11, with a median of 1 (IQR 1–2). Of these home care users, 20 015 (14.4%) died within 6 months of an assessment.
We excluded 8856 assessments (out of 1 365 606) that had missing responses for predictors that were included in our model.
Table 1 provides the characteristics of patients who received home care and were included in our study. The mean (SD) age in our total cohort at the time of a RAI-HC assessment was 79.7 (10.5) years and most were female (65.0%). Among the reported comorbidities, hypertension was the most prevalent condition (60.8%), followed by coronary artery disease (26.8%) and Alzheimer disease or other dementias (23.5%). Other morbidities included in our algorithm (i.e., a history of stroke, congestive heart failure, multiple sclerosis, parkinsonism, cancer, chronic obstructive pulmonary disease [COPD] and renal failure) had a prevalence of less than 20%. A substantial proportion of the population (78.3%) required extensive assistance in performing ordinary housework, with meal preparation and phone use (i.e., tasks related to IADL). Most (55.2%) had no difficulties maintaining their hygiene, using the toilet, or with locomotion or eating (i.e., tasks related to ADL). Only 1.3% of the those who received care had been given a prognosis of having fewer than 6 months to live. However, many had reported worsening capacity to perform ADL (45.9%), cognitive skills (18.9%) and symptoms of health instability, as indicated by having edema (27.1%) or shortness of breath (26.4%) in the past 3 days. Overall, there were no systematic differences between the derivation and validation cohorts.
Baseline characteristics of the derivation and validation cohorts
Cox proportional hazards regression
Hazard ratio estimates from the Cox proportional hazard regressions for the derivation and total cohorts are shown in Appendix 2. In our derivation cohort, predictors with the largest effect on mortality were the diagnosis of an end-stage disease (hazard ratio [HR] 3.22, 95% confidence interval [CI] 3.11–3.33) and limitations in performing ADL (HR 3.58, 95% CI 3.42–3.76 for an ADL Hierarchy Scale score of 6) or IADL (HR 2.02, 95% CI, 1.92–2.13 for an IADL Difficulty Scale score of 6). These were closely followed by a reported history of (nonskin) cancer, particularly among patients who were receiving chemotherapy (HR 3.34, 95% CI 3.21–3.48), congestive heart failure (HR 1.35, 95% CI 1.32–1.37), renal failure and receiving hemodialysis treatment (HR 1.66, 95% CI 1.58–1.74), and chronic obstructive pulmonary disease (COPD) with on-going oxygen therapy (HR 1.54, 95% CI 1.50–1.58). Signs and symptoms of health instability — such as weight loss, shortness of breath, vomiting over 3 consecutive days and other items that make up the Changes in Health, End-Stage Disease, Signs, and Symptoms (CHESS) scale — were all independently and highly predictive of death at 6 months after a RAI-HC assessment.
The top predictors of mortality identified in our regression model were disproportionally represented in the highest risk groups. We found that the proportion of patients who had limitations in function (i.e., IADL and ADL tasks) or cognition, a history of cancer and signs of health instability increased in parallel to the predicted mortality risk. A detailed description of the baseline characteristics for assessments in each of the 61 bins is available in Appendix 7, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200022/tab-related-content.
Model performance
We created a calibration plot to compare the mean predicted probability of death in each risk bin against its observed Kaplan–Meier survival estimates at 6 months after a RAI-HC assessment (Appendix 8, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200022/tab-related-content). The algorithm was well-calibrated across most risk bins. The deviation in our temporal validation on RAI-HC assessments performed in 2013 (i.e., calibration-in-the-large) was 1.46 percentage points on average. The calibration slope was 0.87 (95% CI 0.85–0.90), which represented a nonsignificant concern with over-fitting and good generalizability.19 Discrimination was good in the validation cohort (C statistic 0.753, 95% CI 0.750–0.756).
We subsequently evaluated the calibration across categories of the top predictors of mortality (Appendix 9, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200022/tab-related-content). We evaluated the ADL and IADL scale scores, the presence or history of select diseases and receipt of life-sustaining therapies (i.e., hemodialysis, oxygen therapy and chemotherapy). The algorithm was well-calibrated across most predictors, with less than a 4.8 percentage point deviation from the observed probability of death in all categories. Calibration was poor in patients who were scheduled to receive life-sustaining therapies (e.g., dialysis, ventilation and chemotherapy) but did not receive them, likely because of their unexpected deaths.
Survival
The mean predicted 6-month probability of death in our full study cohort was 10.8% (95% CI 10.7%–10.8%) and ranged from 1.54% in the lowest risk bin (95% CI 1.53%–1.54%) to 98.1% in the highest bin (95% CI 98.1%–98.2%). Median Kaplan–Meier survival time (Figure 2) varied from 28 days (11–84 d at the 25th and 75th percentiles) in the highest risk group to over 8 years (1925 to > 3420 d) in the lowest risk group.
Kaplan–Meier survival estimates of select risk groups in the full cohort.
Interpretation
We developed a predictive model of 6-month mortality among older, community-dwelling adults who required home care using linked personal-level data that included self-reportable chronic conditions, symptoms and level of function. The data represented a true population perspective and are well-suited for the development of prognostication tools for the community setting, given the public funding and provision of homes care services in Ontario.
Among the top predictors of 6-month mortality, we found measures of functional capacity (limitations in IADL and ADL) to be important factors capturing decline and deterioration toward death — even more than most individual chronic diseases themselves. Although only a small number of home care users were given a prognosis of less than 6 months to live, nearly one-quarter of the study cohort exhibited rapid health declines that were correlated with elevated 6-month mortality risk — an indication that a sizeable proportion of this population may have end-of-life care needs that were not identified or met. For clinicians as well as decision-makers, this suggests that RESPECT has the potential to serve as a trigger for a comprehensive palliative care needs assessment after the completion of each RAI-HC assessment.
Overall, the RESPECT model showed good performance. Compared with similar algorithms that estimate short-term mortality risk (≤ 1 year), RESPECT had comparable discrimination (C statistic of 0.75 compared with 0.66 to 0.72)20–23 but improved calibration and classification of risk according to incremental survival thresholds. For example, 1 well-known algorithm predicting mortality using data elements from the RAI-HC is the CHESS scale. A 2016 study20 using a similar cohort of patients who received home care recipients in Ontario reported poorer discrimination (C statistic of 0.66 for the CHESS scale) than RESPECT. Furthermore, the CHESS scale does not present meaningful stratifications according to survival time, because the top 2 risk categories of the CHESS scale represent an incremental increase in predicted risk of 49% (from a 37% 6-month mortality risk when the patient scores 4 out of 5 on the scale to 86% 6-month mortality risk associated with the highest risk score of 5 out of 5), despite showing similar calibration (calibration plot slope of 0.89 and intercept of 0.014) as for RESPECT (Appendix 10, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.200022/tab-related-content). As patients with CHESS scores of 4 or 5 comprise 3.0% and 0.2% of the population, respectively, these translate to differences of 200–3000 patients per 100 0000 home care users who may require palliative and end-of-life care. The large, 10-fold difference poses a practical challenge to health system planning. In contrast, differences among the highest risk bins in RESPECT represented incremental survival of 3 weeks that translated into a constant increase by intervals of 0.2% of the population, which may be more easily operationalized for capacity planning at the health system level.
Limitations
As with many prediction models, RESPECT is less well-calibrated at the extremes of the distribution. In particular, we found that RESPECT overpredicted the mortality risk of patients in our top 3 risk bins. The imprecision may reflect the absence of other potentially influential predictors (e.g., cancer type and staging information) or interaction effects in our model. Despite this, clinically, these risk groups captured patients with extremely poor prognosis, all of whom could benefit from palliative care. Most of the patients (94%, 89% and 84%, respectively) in these 3 bins died within 1 year of their RAI-HC assessment. Nevertheless, we are currently exploring other modelling approaches that may better account for nonlinear and possible time-varying effects to improve the predictive performance of RESPECT. At the inception of this study, we did not have the opportunity to perform an external, geographical validation of the final model. The validation was performed among home care users in Ontario. We aim to further assess our algorithm’s generalizability by validating it in other Canadian provinces where RAI-HC data are available.
Conclusion
We developed RESPECT from routinely collected health care administrative data available in many Canadian provinces. Our model provides useful survival information that can inform when palliative care should be started. It could be readily implemented at the health system level as a care planning tool and at the patient level as a risk communication aid. The tool utilizes information from the RAI-HC that can be easily self-reported by patients or their caregivers to support their decision-making and, potentially, allows them to advocate for their care needs to achieve the appropriate balance of life-prolonging and comfort care.
Footnotes
Competing interests: Spruin has received support as a research assistant from Bruyère Research Institute. No other competing interests were declared.
This article has been peer reviewed.
Contributors: Amy Hsu and Douglas Manuel were co-leads in the study design, and were responsible for the interpretation of the results and writing of the manuscript. Sara Spruin, Mathieu Chalifoux, Robert Talarico and Yulric Sequeira were responsible for the analysis and interpretation of data, and for providing critical revisions to the manuscript for its methodological content. Carol Bennett, Monica Taljaard, Daniel Kobewka, Susan Bronskill, Andrew Costa and Sarah Beach contributed to the design of this study and provided critical reviews of its clinical and methodological content. Peter Tanuseputro was responsible for the inception of this project, for the study design and critical revisions to the manuscript for important intellectual content. All of the authors reviewed the manuscript, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.
Funding: This study was supported by ICES through an Applied Health Research Question (AHRQ) grant (project no. 2015 0901 074 000). This study also received funding from the Canadian Institutes of Health Research (funding reference no. PJT–153251) and the Bruyère Big Data Research Program.
Data sharing: The data set from this study is held securely in coded form at ICES. Although legal data-sharing agreements between ICES and data providers (e.g., health care organizations and government) prohibit ICES from making the data set publicly available, access may be granted to those who meet prespecified criteria for confidential access, available at https://www.ices.on.ca/DAS (email: das{at}ices.on.ca). The full data set creation plan and underlying analytic code are available from the authors upon request, understanding that the computer programs may rely upon coding templates or macros that are unique to ICES and are therefore either inaccessible or may require modification.
Disclaimer: This study was supported by ICES, which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred.
- Accepted February 24, 2021.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons.org/licenses/by-nc-nd/4.0/
References
In this issue
Article tools
Jump to section
Related Articles
Cited By...
More in this TOC Section
Similar Articles
Collections
