Skip to main content

Main menu

  • Home
  • Content
    • Current issue
    • Past issues
    • Early releases
    • Collections
    • Sections
    • Blog
    • Infographics & illustrations
    • Podcasts
    • COVID-19 Articles
  • Authors
    • Overview for authors
    • Submission guidelines
    • Submit a manuscript
    • Forms
    • Editorial process
    • Editorial policies
    • Peer review process
    • Publication fees
    • Reprint requests
    • Open access
  • CMA Members
    • Overview for members
    • Earn CPD Credits
    • Print copies of CMAJ
  • Subscribers
    • General information
    • View prices
  • Alerts
    • Email alerts
    • RSS
  • JAMC
    • À propos
    • Numéro en cours
    • Archives
    • Sections
    • Abonnement
    • Alertes
    • Trousse média 2022
  • CMAJ JOURNALS
    • CMAJ Open
    • CJS
    • JAMC
    • JPN

User menu

Search

  • Advanced search
CMAJ
  • CMAJ JOURNALS
    • CMAJ Open
    • CJS
    • JAMC
    • JPN
CMAJ

Advanced Search

  • Home
  • Content
    • Current issue
    • Past issues
    • Early releases
    • Collections
    • Sections
    • Blog
    • Infographics & illustrations
    • Podcasts
    • COVID-19 Articles
  • Authors
    • Overview for authors
    • Submission guidelines
    • Submit a manuscript
    • Forms
    • Editorial process
    • Editorial policies
    • Peer review process
    • Publication fees
    • Reprint requests
    • Open access
  • CMA Members
    • Overview for members
    • Earn CPD Credits
    • Print copies of CMAJ
  • Subscribers
    • General information
    • View prices
  • Alerts
    • Email alerts
    • RSS
  • JAMC
    • À propos
    • Numéro en cours
    • Archives
    • Sections
    • Abonnement
    • Alertes
    • Trousse média 2022
  • Visit CMAJ on Facebook
  • Follow CMAJ on Twitter
  • Follow CMAJ on Pinterest
  • Follow CMAJ on Youtube
  • Follow CMAJ on Instagram
Research

Validation of quality indicators for end-of-life communication: results of a multicentre survey

Daren K. Heyland, Peter Dodek, John J. You, Tasnim Sinuff, Tim Hiebert, Carolyn Tayler, Xuran Jiang, Jessica Simon and James Downar; for the ACCEPT Study Team and the Canadian Researchers at the End of Life Network (CARENET)
CMAJ July 31, 2017 189 (30) E980-E989; DOI: https://doi.org/10.1503/cmaj.160515
Daren K. Heyland
Department of Critical Care Medicine (Heyland), Kingston General Hospital; Department of Public Health Sciences (Heyland), Queen’s University; Clinical Evaluation Research Unit (Heyland, Jiang), Kingston General Hospital, Kingston, Ont.; Centre for Health Evaluation and Outcome Sciences and Division of Critical Care Medicine (Dodek), St. Paul’s Hospital and University of British Columbia, Vancouver, BC; Departments of Medicine, and Health Research Methods, Evidence and Impact (You), McMaster University, Hamilton, Ont.; Department of Critical Care Medicine, Sunnybrook Hospital and Sunnybrook Research Institute; Interdepartmental Division of Critical Care Medicine and Department of Medicine, University of Toronto (Sinuff), Toronto, Ont.; Winnipeg Regional Health Authority Palliative Care Program (Hiebert), Winnipeg, Man.; Fraser Health Authority (Tayler), Surrey, BC; Division of Palliative Medicine (Simon), Department of Oncology, Department of Community Health Sciences and Department of Medicine, University of Calgary, Alta.; Divisions of Critical Care and Palliative Care, Department of Medicine (Downar), University of Toronto, Toronto, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: dkh2@queensu.ca
Peter Dodek
Department of Critical Care Medicine (Heyland), Kingston General Hospital; Department of Public Health Sciences (Heyland), Queen’s University; Clinical Evaluation Research Unit (Heyland, Jiang), Kingston General Hospital, Kingston, Ont.; Centre for Health Evaluation and Outcome Sciences and Division of Critical Care Medicine (Dodek), St. Paul’s Hospital and University of British Columbia, Vancouver, BC; Departments of Medicine, and Health Research Methods, Evidence and Impact (You), McMaster University, Hamilton, Ont.; Department of Critical Care Medicine, Sunnybrook Hospital and Sunnybrook Research Institute; Interdepartmental Division of Critical Care Medicine and Department of Medicine, University of Toronto (Sinuff), Toronto, Ont.; Winnipeg Regional Health Authority Palliative Care Program (Hiebert), Winnipeg, Man.; Fraser Health Authority (Tayler), Surrey, BC; Division of Palliative Medicine (Simon), Department of Oncology, Department of Community Health Sciences and Department of Medicine, University of Calgary, Alta.; Divisions of Critical Care and Palliative Care, Department of Medicine (Downar), University of Toronto, Toronto, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
John J. You
Department of Critical Care Medicine (Heyland), Kingston General Hospital; Department of Public Health Sciences (Heyland), Queen’s University; Clinical Evaluation Research Unit (Heyland, Jiang), Kingston General Hospital, Kingston, Ont.; Centre for Health Evaluation and Outcome Sciences and Division of Critical Care Medicine (Dodek), St. Paul’s Hospital and University of British Columbia, Vancouver, BC; Departments of Medicine, and Health Research Methods, Evidence and Impact (You), McMaster University, Hamilton, Ont.; Department of Critical Care Medicine, Sunnybrook Hospital and Sunnybrook Research Institute; Interdepartmental Division of Critical Care Medicine and Department of Medicine, University of Toronto (Sinuff), Toronto, Ont.; Winnipeg Regional Health Authority Palliative Care Program (Hiebert), Winnipeg, Man.; Fraser Health Authority (Tayler), Surrey, BC; Division of Palliative Medicine (Simon), Department of Oncology, Department of Community Health Sciences and Department of Medicine, University of Calgary, Alta.; Divisions of Critical Care and Palliative Care, Department of Medicine (Downar), University of Toronto, Toronto, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tasnim Sinuff
Department of Critical Care Medicine (Heyland), Kingston General Hospital; Department of Public Health Sciences (Heyland), Queen’s University; Clinical Evaluation Research Unit (Heyland, Jiang), Kingston General Hospital, Kingston, Ont.; Centre for Health Evaluation and Outcome Sciences and Division of Critical Care Medicine (Dodek), St. Paul’s Hospital and University of British Columbia, Vancouver, BC; Departments of Medicine, and Health Research Methods, Evidence and Impact (You), McMaster University, Hamilton, Ont.; Department of Critical Care Medicine, Sunnybrook Hospital and Sunnybrook Research Institute; Interdepartmental Division of Critical Care Medicine and Department of Medicine, University of Toronto (Sinuff), Toronto, Ont.; Winnipeg Regional Health Authority Palliative Care Program (Hiebert), Winnipeg, Man.; Fraser Health Authority (Tayler), Surrey, BC; Division of Palliative Medicine (Simon), Department of Oncology, Department of Community Health Sciences and Department of Medicine, University of Calgary, Alta.; Divisions of Critical Care and Palliative Care, Department of Medicine (Downar), University of Toronto, Toronto, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Tim Hiebert
Department of Critical Care Medicine (Heyland), Kingston General Hospital; Department of Public Health Sciences (Heyland), Queen’s University; Clinical Evaluation Research Unit (Heyland, Jiang), Kingston General Hospital, Kingston, Ont.; Centre for Health Evaluation and Outcome Sciences and Division of Critical Care Medicine (Dodek), St. Paul’s Hospital and University of British Columbia, Vancouver, BC; Departments of Medicine, and Health Research Methods, Evidence and Impact (You), McMaster University, Hamilton, Ont.; Department of Critical Care Medicine, Sunnybrook Hospital and Sunnybrook Research Institute; Interdepartmental Division of Critical Care Medicine and Department of Medicine, University of Toronto (Sinuff), Toronto, Ont.; Winnipeg Regional Health Authority Palliative Care Program (Hiebert), Winnipeg, Man.; Fraser Health Authority (Tayler), Surrey, BC; Division of Palliative Medicine (Simon), Department of Oncology, Department of Community Health Sciences and Department of Medicine, University of Calgary, Alta.; Divisions of Critical Care and Palliative Care, Department of Medicine (Downar), University of Toronto, Toronto, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Carolyn Tayler
Department of Critical Care Medicine (Heyland), Kingston General Hospital; Department of Public Health Sciences (Heyland), Queen’s University; Clinical Evaluation Research Unit (Heyland, Jiang), Kingston General Hospital, Kingston, Ont.; Centre for Health Evaluation and Outcome Sciences and Division of Critical Care Medicine (Dodek), St. Paul’s Hospital and University of British Columbia, Vancouver, BC; Departments of Medicine, and Health Research Methods, Evidence and Impact (You), McMaster University, Hamilton, Ont.; Department of Critical Care Medicine, Sunnybrook Hospital and Sunnybrook Research Institute; Interdepartmental Division of Critical Care Medicine and Department of Medicine, University of Toronto (Sinuff), Toronto, Ont.; Winnipeg Regional Health Authority Palliative Care Program (Hiebert), Winnipeg, Man.; Fraser Health Authority (Tayler), Surrey, BC; Division of Palliative Medicine (Simon), Department of Oncology, Department of Community Health Sciences and Department of Medicine, University of Calgary, Alta.; Divisions of Critical Care and Palliative Care, Department of Medicine (Downar), University of Toronto, Toronto, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Xuran Jiang
Department of Critical Care Medicine (Heyland), Kingston General Hospital; Department of Public Health Sciences (Heyland), Queen’s University; Clinical Evaluation Research Unit (Heyland, Jiang), Kingston General Hospital, Kingston, Ont.; Centre for Health Evaluation and Outcome Sciences and Division of Critical Care Medicine (Dodek), St. Paul’s Hospital and University of British Columbia, Vancouver, BC; Departments of Medicine, and Health Research Methods, Evidence and Impact (You), McMaster University, Hamilton, Ont.; Department of Critical Care Medicine, Sunnybrook Hospital and Sunnybrook Research Institute; Interdepartmental Division of Critical Care Medicine and Department of Medicine, University of Toronto (Sinuff), Toronto, Ont.; Winnipeg Regional Health Authority Palliative Care Program (Hiebert), Winnipeg, Man.; Fraser Health Authority (Tayler), Surrey, BC; Division of Palliative Medicine (Simon), Department of Oncology, Department of Community Health Sciences and Department of Medicine, University of Calgary, Alta.; Divisions of Critical Care and Palliative Care, Department of Medicine (Downar), University of Toronto, Toronto, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jessica Simon
Department of Critical Care Medicine (Heyland), Kingston General Hospital; Department of Public Health Sciences (Heyland), Queen’s University; Clinical Evaluation Research Unit (Heyland, Jiang), Kingston General Hospital, Kingston, Ont.; Centre for Health Evaluation and Outcome Sciences and Division of Critical Care Medicine (Dodek), St. Paul’s Hospital and University of British Columbia, Vancouver, BC; Departments of Medicine, and Health Research Methods, Evidence and Impact (You), McMaster University, Hamilton, Ont.; Department of Critical Care Medicine, Sunnybrook Hospital and Sunnybrook Research Institute; Interdepartmental Division of Critical Care Medicine and Department of Medicine, University of Toronto (Sinuff), Toronto, Ont.; Winnipeg Regional Health Authority Palliative Care Program (Hiebert), Winnipeg, Man.; Fraser Health Authority (Tayler), Surrey, BC; Division of Palliative Medicine (Simon), Department of Oncology, Department of Community Health Sciences and Department of Medicine, University of Calgary, Alta.; Divisions of Critical Care and Palliative Care, Department of Medicine (Downar), University of Toronto, Toronto, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
James Downar
Department of Critical Care Medicine (Heyland), Kingston General Hospital; Department of Public Health Sciences (Heyland), Queen’s University; Clinical Evaluation Research Unit (Heyland, Jiang), Kingston General Hospital, Kingston, Ont.; Centre for Health Evaluation and Outcome Sciences and Division of Critical Care Medicine (Dodek), St. Paul’s Hospital and University of British Columbia, Vancouver, BC; Departments of Medicine, and Health Research Methods, Evidence and Impact (You), McMaster University, Hamilton, Ont.; Department of Critical Care Medicine, Sunnybrook Hospital and Sunnybrook Research Institute; Interdepartmental Division of Critical Care Medicine and Department of Medicine, University of Toronto (Sinuff), Toronto, Ont.; Winnipeg Regional Health Authority Palliative Care Program (Hiebert), Winnipeg, Man.; Fraser Health Authority (Tayler), Surrey, BC; Division of Palliative Medicine (Simon), Department of Oncology, Department of Community Health Sciences and Department of Medicine, University of Calgary, Alta.; Divisions of Critical Care and Palliative Care, Department of Medicine (Downar), University of Toronto, Toronto, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Tables
  • Related Content
  • Responses
  • Metrics
  • PDF
Loading

Abstract

BACKGROUND: The lack of validated quality indicators is a major barrier to improving end-of-life communication and decision-making. We sought to show the feasibility of and provide initial validation for a set of quality indicators related to end-of-life communication and decision-making.

METHODS: We administered a questionnaire to patients and their family members in 12 hospitals and asked them about advance care planning and goals-of-care discussions. Responses were used to calculate a quality indicator score. To validate this score, we determined its correlation with the concordance between the patients’ expressed wishes and the medical order for life-sustaining treatments recorded in the hospital chart. We compared the correlation with concordance for the advance care planning component score with that for the goal-of-care discussion scores.

RESULTS: We enrolled 297 patients and 209 family members. At all sites, both overall quality indicators and individual domain scores were low and there was wide variability around the point estimates. The highest-ranking institution had an overall quality indicator score (95% confidence interval) of 40% (36%–44%) and the lowest had a score of 18% (11%–25%). There was a strong correlation between the overall quality indicator score and the concordance measure (r = 0.72, p = 0.008); the estimated correlation between the advance care planning score and the concordance measure (r = 0.35) was weaker than that between the goal-of-care discussion scores and the concordance measure (r = 0.53).

INTERPRETATION: Quality of end-of-life communication and decision-making appears low overall, with considerable variability across hospitals. The proposed quality indicator measure shows feasibility and partial validity. Study registration: ClinicalTrials.gov, no. NCT01362855

Despite numerous calls to improve end-of-life care, and communication and decision-making in particular,1–4 over 2 decades there has been little progress in this arena.5,6 One of the major reasons for this limited progress may be the lack of quality indicators that are meaningful at a health system level to guide decision-makers as they introduce policies and practices to improve end-of-life care.7,8 We recently published a study that used a modified Delphi consensus process to develop a set of quality indicators related to advance care planning, goals-of-care discussions and documentation of these plans and goals (Table 1).9 Quality indicators should be evaluated based on their reliability, validity, responsiveness to change and ease of implementation.10 Because of the robust methodology used in developing these quality indicators,9 we believe they are clinically sensible and have face and content validity. Here we describe implementation of these quality indicators in a multicentre study to evaluate their feasibility and construct validity. A secondary objective was to establish a method for ranking institutions with respect to their performance in end-of-life–related communication and decision-making.

View this table:
  • View inline
  • View popup
Table 1:

List of quality indicators

Methods

This work was guided by a conceptual model of communication and decision-making in seriously ill and older patients who had been admitted to hospital, and the impact of these processes on patient-centred outcomes (Figure 1).9 To validate the aforementioned quality indicators, we conducted a multicentre, prospective audit of communication and decision-making related to the use or nonuse of life-sustaining treatments during serious illness, from the perspectives of patients and their family members; this was called the Audit of Communication, Care Planning, and Documentation (ACCEPT) Study.12 The association between our conceptual model and the ACCEPT audit is illustrated by the box in Figure 1 (downstream medical care actually received and satisfaction with hospital care were not evaluated).

Figure 1:
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 1:

Conceptual model of improving end-of-life communication and decision-making, showing Audit of Communication, Care Planning, and Documentation (ACCEPT) Study quality indicators.9 In this model, we propose that advance care planning should ideally occur before hospital admission and inform goals-of-care discussions that occur within hospital. Ultimately, these plans and goals are translated into written documents or medical orders for the use or nonuse of life-sustaining treatments. For medical care at end of life to be consistent with patient values and preferences (the primary outcome), medical orders documented within the hospital records (surrogate outcome) must be concordant with these expressed preferences. Ultimately, the processes of communication and decision-making affect patient and family-centred outcomes, such as knowledge, stress, anxiety, other measures of psychological well-being and overall satisfaction. Given that advance care planning should occur before serious illness or admission to hospital, admission is an opportunity to assess satisfaction with previous advance care planning conversations. Similarly, discharge from hospital is an opportunity to measure patient satisfaction with goal-of-care discussion and decision-making.

The method for the ACCEPT audit has been described previously.1 In brief, inpatients from 12 teaching hospitals in Ontario, Manitoba, Alberta and British Columbia, Canada, were screened for eligibility (Appendix 1, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.160515/-/DC1). Patients were enrolled if they had advanced medical diseases (Appendix 2, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.160515/-/DC1) or if they were 80 years of age or older. If none of the above criteria were met, we also included any patient whose death within the next 6 months would not surprise any member of their care team.13

We approached consecutive, eligible patients and their family members for their written consent. Patients who were unable to communicate owing to language barriers (English or French only) or cognitive reasons were excluded, but if their family member was available and able to communicate, they were approached independently. Enrolled patients were asked to identify an adult family member who knew them best (inclusive of partners, significant others or close friends) and who had visited the patient in hospital at least once. We approached the patient 2 to 5 days after admission to allow for abatement of symptoms present at the time of admission and to allow for participation of both patient and family members.

Upon enrolment, the research assistant conducted separate face-to-face interviews with patients and family members using the previously validated ACCEPT questionnaire.14 In brief, we assessed respondents’ engagement in advance care planning before they were admitted to hospital and then asked questions about whether key steps related to goal-of-care discussion had occurred during the current hospital admission. These questions related to advance care planning and goal-of-care discussion were based on our previously developed quality indicators.9 To elicit preferences about use of life-sustaining treatments, we used the following taxonomy: respondents were asked to select the one that best represented their preferences: 1) Use machines and all possible measures including cardiopulmonary resuscitation (CPR) with a focus on keeping me (or my family member) alive at all costs; 2) Use machines and all possible measures with a focus on keeping me (or my family member) alive, but if my heart stops, no CPR; 3) Use machines only in the short term to see if I (or my family member) will get better, but if my illness is prolonged, change focus to comfort measures only. If the heart stops, no CPR; 4) Use full medical care to prolong my life (or life of my family member), but if my heart or breathing stops, no CPR or breathing machines; 5) Use comfort measures only, with a focus on improving quality of life and comfort. Allow natural death and no artificial prolongation of life and no CPR; or 6) Unsure.

We also assessed satisfaction with end-of-life care, including communication and decision-making, in the 4 weeks before hospital admission through the use of the validated Canadian Health Care Evaluation Project (CANHELP) Questionnaire.15 The CANHELP instrument is a questionnaire that quantifies patients’ and family caregivers’ satisfaction on a 5-point Likert scale.

After completion of the ACCEPT questionnaire, the research nurse reviewed the hospital record to document the medical orders for the use of life-sustaining treatments and items from the documentation domain of the quality indicator framework.9

Statistical analysis

Demographic characteristics of patients and their family members are described as means, standard deviations and ranges for continuous variables, and as counts and percentages for categorical variables. Each quality indicator belonged to one of the advance care planning, goal-of-care discussion or documentation domains. Item scores are the proportion of patients (or family members) who affirmed that each quality indicator listed in Table 1 had occurred, multiplied by a weighting. Weights for each item were derived from the importance ratings from our previous work on the development of these quality indicators (Table 1).9 The overall quality indicator score, known as the ACCEPT quality indicator score, is the raw sum of all quality indicator items divided by the raw sum of highest possible values of these items (on a scale of 0% to 100%). We combined both patients’ and family members’ responses to develop average site overall and domain scores. We evaluated the internal consistency of the overall and domain scores using the Cronbach’s α.

To provide a measure of validity for this quality indicator scoring approach, a priori, we hypothesized that the overall ACCEPT quality indicator score would correlate with concordance between stated preferences and documented goals of care; concordance was calculated only for participants who had both of these items available. Patients or family members who had missing data or who were “unsure” of their preferences were excluded from these analyses. If the patient or family member preferred CPR and if there were no documented goals of care on the chart, the case was considered concordant because the default in all participating institutions was resuscitation. If patients or family members preferred not to be resuscitated and there was no documentation, such cases were considered to be discordant and consistent with a medical error. This concordance measure is one of the quality indicators in the documentation domain but was excluded from this domain, to stand alone as an outcome (Figure 1) for this validation exercise.

Because the goal-of-care discussion items were more likely to influence the concordance statistic in hospital, we postulated that the advance care planning domain score (a measure of prehospital activity) would correlate less strongly with concordance than the goal-of-care discussion domain scores. Moreover, because the CANHELP satisfaction measure was administered early in the hospital course, we postulated that the pertinent domains and items from the CANHELP questionnaire relevant to advance care planning would correlate with the advance care planning domain of the quality indicators. Associations between the whole or individual domains of the quality indicator score, the concordance measure, and the items and domains of the CANHELP scores were assessed by Pearson correlation coefficients.

Ethics approval

The research ethics boards of all participating institutions approved the study.

Results

We approached 580 eligible patients and 318 eligible family members; 297 patients and 209 family members consented, yielding an enrolment rate of 51% and 66%, respectively (Figure 2). The average age of patients was 80 years and 43% lived alone (Table 2). The average age of family members was 61 years; most were children of the patient (55%; Table 3). Most patients and family members were white and spoke either English or French. Hospital site characteristics are provided in Table 4.

Figure 2:
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 2:

Patient flow diagram. Patients or family members who are excluded may fall into multiple categories of exclusion. ACCEPT = Audit of Communication, Care Planning, and Documentation study.

View this table:
  • View inline
  • View popup
Table 2:

Patient characteristics

View this table:
  • View inline
  • View popup
Table 3:

Family member characteristics

View this table:
  • View inline
  • View popup
Table 4:

Site characteristics

Overall, all quality indicator scores were low, with wide confidence limits around these scores, and variability in performance among institutions (Table 5). Nevertheless, quality indicator scores for the highest-ranked hospital did not overlap with those from the lowest-ranked hospital; the highest-ranked hospital had an overall score (95% confidence interval [CI]) of 40% (36%–44%) and the lowest-ranked hospital had a score of 18% (11%–25%).

View this table:
  • View inline
  • View popup
Table 5:

Scores and ranking of quality indicators by hospital*

Concordance at a hospital level ranged from 6% to 65%. Overall, there was a strong and statistically significant correlation between the overall ACCEPT quality indicator score and concordance (Table 6; r = 0.72, p = 0.008). As expected, the correlations between the advance care planning domain, and concordance and overall quality indicator score were weaker than the correlations between the goal-of-care discussion domain and concordance (Table 6). There were no statistically significant correlations between the advance care planning quality indicator domain score and different aspects of the CANHELP satisfaction questionnaire (overall score, domain scores or selected relevant item scores; Table 7). The Cronbach’s α for the overall score was 0.82, and for the advance care planning, goal-of-care discussion and documentation domains, scores were 0.64, 0.82 and 0.37, respectively. The low scores in the documentation domain were driven largely by the third and fifth items; when these items were excluded, the Cronbach’s α was 0.77 and the correlation with the overall quality indicator score remained relatively unchanged (r = 0.71, p = 0.01).

View this table:
  • View inline
  • View popup
Table 6:

Associations between site ACCEPT quality indicator score and concordance measures (n = 12)*

View this table:
  • View inline
  • View popup
Table 7:

Associations between site advance care planning domain score and site CANHELP scores (n = 12 sites)*

Interpretation

In this multicentre audit of end-of-life communication and decision-making practices, we implemented a quality indicator measurement framework to show the feasibility of multisite measurements, evaluate its internal consistency, add further construct validation to the quality indicator scoring system, and discriminate among sites based on the quality of their end-of-life communication and decision-making. The purpose of these quality indicators and performance rankings is to improve quality of care by emphasizing best practices, ensuring accountability of providers and stimulating healthy competition.16

Importantly, our analysis was based on an a priori conceptual framework. In this framework, we positioned concordance between a patient’s expressed wishes and documented orders for life-sustaining treatments as the reference measure. We observed that our overall quality indicator score correlated strongly with this measure and, consistent with our hypothesis, the correlations between the goal-of-care discussion domain and the concordance measure were greater than the correlation between the advance care planning domain and the concordance measure. Although we observed significant variability in hospital performance, overall and domain quality indicator scores were low. Notwithstanding this fact, we were able to discriminate the highest-ranked hospitals from the lowest-ranked hospitals.

We and others have previously documented substantial quality concerns related to end-of-life communication and decision-making in patients in hospital.1–6 Measuring quality indicators related to these important processes may be a stimulus for improvement. However, few health care organizations routinely measure the quality of end-of-life care in general, and even fewer routinely evaluate the quality of communication and decision-making.8 Recently, some quality indicators were developed in the broad field of palliative/end-of-life care.17,19,20 Some of these include aspects of communication and decision-making, but these domains are not the focus of these measures. Accordingly, we developed9 and have now validated a novel set of quality indicators that are specifically related to end-of-life communication and decision-making.

Validation of quality indicators is done only infrequently.21 In our initial attempt to validate our quality indicators, we postulated that prehospital communication and decision-making (advance care planning) would have less impact on overall quality of communication and decision-making than in-hospital activities (goal-of-care discussion and documentation), as judged by the correlation of these domains with concordance between stated preferences and documented goals of care. Indeed, this is exactly what we found. However, we could not show any association between prehospital advance care planning activities and a measure of satisfaction (CANHELP questionnaire) used at admission to hospital (and meant to reflect the satisfaction with prehospital activities over the last 4 weeks). This weakens our attempt to provide some validation to the quality indicator measures. However, it may be that advance care planning activities conducted in the community do not influence satisfaction (as this is not the primary objective of advance care planning), or that satisfaction with advance care planning communication that had occurred before the 4 weeks prior to hospital admission was not captured by the time-limited framing of the CANHELP questionnaire.

Our findings do not mean that community-based advance care planning activities are not relevant for in-hospital goal-of-care discussion or documentation; they contribute by preparing patients for in-hospital communication and decision-making.12,18 But these complementary prehospital advance care planning activities are clearly not a substitute for goal-of-care discussion and documentation that should occur at the time of an admission to hospital. Furthermore, the lack of a strong association between advance care planning and our overall quality indicator score and concordance measure may, in part, be due to the disconnect between what happens in the community and what happens in the hospital. In health care systems where primary care is more directly related and integrated into acute care21 (unlike the current situation in Canada), there may be a stronger correlation between advance care planning and the overall quality indicator score and the concordance measure. To simplify the measurement process, some hospitals could omit the advance care planning domain from the measurement framework and focus their time and resources on the domains that they are better positioned to address (goal-of-care discussion). We also acknowledge that although our weights were empirically derived, they may not be reflective of the constituents served by other hospitals. In considering the adoption of this measurement framework to local hospitals, these weights could be either re-evaluated or omitted all together. Finally, we suggest that users omit the third and fifth items from the documentation domain to achieve results with higher internal consistency.

Limitations

One limitation of this study is the small sample size. The limited number of participants from each site contributes to imprecision with respect to the estimates of site scores. In addition, all of the participating hospitals were teaching hospitals; therefore, our findings may not be generalizable to nonteaching hospitals. We also acknowledge that our initial validation work is limited to hospital-based outcomes. We did not evaluate the association between our quality indicators and long-term outcomes. Finally, we did not assess the reliability of these quality indicators; further studies need to include this type of psychometric evaluation.

Conclusions

In conclusion, we have developed and validated quality indicators that can be used to measure the quality of end-of-life communication and decision-making. The results of this analysis suggest that it is feasible to implement these quality indicators in a variety of hospitals, that they are internally consistent and have construct validity, and that they can discriminate between highest-and lowest-ranked hospitals. Widespread adoption of this quality indicator framework could assist in process improvement for individual hospitals and improve the experience of patients who have serious illnesses, and that of their family members.

Footnotes

  • See related article at www.cmaj.ca/lookup/doi/10.1503/cmaj.170280

  • Competing interests: Jessica Simon reports grants from Alberta Innovates Health Solutions and personal fees from Alberta Health Services, during the conduct of the study. James Downar reports receiving fees for statistical analysis from Toronto General/Toronto Western Hospital Foundation and personal fees from Associated Medical Services Inc., during the conduct of the study. No other competing interests were declared.

  • This article has been peer reviewed.

  • Contributors: Xuran Jiang conducted the analysis. All authors participated in the acquisition of data, the interpretation of results and the drafting of the manuscript, and approved the final version to be published.

  • Funding: This study was supported by funding from the Canadian Institutes of Health Research, The Michael Smith Health Services Research Foundation in British Columbia, Alberta Innovates, and the Alternative Funding Plan Innovation Fund in Ontario. The funders had no role in the design, interpretation of results or decision to publish.

  • Accepted March 3, 2017.

References

  1. ↵
    1. Heyland DK,
    2. Barwich D,
    3. Pichora D,
    4. et al.
    ACCEPT (Advance Care Planning Evaluation in Elderly Patients) study team; Canadian Researchers at the End of Life Network (CARENET). Failure to engage hospitalized elderly patients and their families in advance care planning. JAMA Intern Med 2013;173:778–87.
    OpenUrlCrossRefPubMed
    1. Field MJ,
    2. Cassel CK
    , editors. Approaching death: improving care at the end of life. Washington: National Academy Press; 1997.
    1. Carstairs S,
    2. Beaudoin GA
    . Quality end of life care: the right of every Canadian. Parliament of Canada; 2000.
  2. ↵
    1. Carstairs S
    . Still not there. Quality end-of-life care: a progress report. Ottawa: Canadian Hospice Palliative Care Association; 2005.
  3. ↵
    The 2015 Quality of Death Index: ranking palliative care across the world. London/NewYork/Hong Kong: The Economist Intelligence Unit; 2015. Available: www.eiuperspectives.economist.com/healthcare/2015-quality-death-index (accessed 2016 Apr. 19).
  4. ↵
    1. Teno JM,
    2. Freedman VA,
    3. Kasper JD,
    4. et al
    . Is care for the dying improving in the United States? J Palliat Med 2015;18:662–6.
    OpenUrlCrossRefPubMed
  5. ↵
    1. Curtis JR,
    2. Engelberg RA
    . Measuring success of interventions to improve the quality of end-of-life care in the intensive care unit. Crit Care Med 2006;34(Suppl):S341–7.
    OpenUrlCrossRefPubMed
  6. ↵
    1. Weiss A,
    2. Downar J
    . Ontario hospitals are not using palliative care performance indicators in their balanced scorecards. J Pain Symptom Manage 2013;46:e1–5.
    OpenUrlCrossRefPubMed
  7. ↵
    1. Sinuff T,
    2. Dodek P,
    3. You JJ,
    4. et al
    . Improving end-of-life communication and decision making: the development of a conceptual framework and quality indicators. J Pain Symptom Manage 2015;49:1070–80.
    OpenUrlCrossRefPubMed
  8. ↵
    1. Stelfox HT,
    2. Straus SE
    . Measuring quality of care: considering conceptual approaches to quality indicator development and evaluation. J Clin Epidemiol 2013;66:1328–37.
    OpenUrlPubMed
    1. Howard M,
    2. Bernard C,
    3. Tan A,
    4. et al
    . Advance care planning: let’s start sooner. Can Fam Physician 2015;61:663–5.
    OpenUrlFREE Full Text
  9. ↵
    Canadian Researchers at the End of Life Network [homepage]. Kingston (ON). Available: www.thecarenet.ca/784-researchers (accessed 2017 July 11).
  10. ↵
    1. Pattison M,
    2. Romer AL
    . Improving care through the end of life: launching a primary care clinic-based program. J Palliat Med 2001;4:249–54.
    OpenUrlCrossRefPubMed
  11. ↵
    1. Heyland DK,
    2. Pichora D,
    3. Dodek P,
    4. et al
    . The development and validation of a questionnaire to audit advance care planning. J Palliative Care Med 2012;2:119.
    OpenUrl
  12. ↵
    1. Heyland DK,
    2. Cook DJ,
    3. Rocker GM,
    4. et al.
    Canadian Researchers at the End of Life Network. The development and validation of a novel questionnaire to measure patient and family satisfaction with end-of-life care: the Canadian Health Care Evaluation Project (CANHELP) questionnaire. Palliat Med 2010;24:682–95.
    OpenUrlCrossRefPubMed
  13. ↵
    1. Adab P,
    2. Rouse AM,
    3. Mohammed MA,
    4. et al
    . Performance league tables: the NHS deserves better. BMJ 2002;324:95–8.
    OpenUrlFREE Full Text
  14. ↵
    1. De Roo ML,
    2. Leemans K,
    3. Claessen SJ,
    4. et al.
    EURO IMPACT. Quality indicators for palliative care: update of a systematic review. J Pain Symptom Manage 2013; 46:556–72.
    OpenUrlCrossRefPubMed
  15. ↵
    1. Sudore RL,
    2. Fried TR
    . Redefining the “planning” in advance care planning: preparing for end-of-life decision making. Ann Intern Med 2010;153:256–61.
    OpenUrlCrossRefPubMed
  16. ↵
    1. Pasman HR,
    2. Brandt HE,
    3. Deliens L,
    4. et al
    . Quality indicators for palliative care: a systematic review. J Pain Symptom Manage 2009;38:145–56.
    OpenUrlCrossRefPubMed
  17. ↵
    1. Dy SM,
    2. Kiley KB,
    3. Ast K,
    4. et al
    . Measuring what matters: top-ranked quality indicators for hospice and palliative care from the American Academy of Hospice and Palliative Medicine and Hospice and Palliative Nurses Association. J Pain Symptom Manage 2015;49:773–81.
    OpenUrlCrossRefPubMed
  18. ↵
    1. Hammes BJ,
    2. Rooney BL,
    3. Gundrum JD
    . A comparative, retrospective, observational study of the prevalence, availability, and specificity of advance care plans in a county that implemented an advance care planning microsystem. J Am Geriatr Soc 2010;58:1249–55.
    OpenUrlCrossRefPubMed
PreviousNext
Back to top

In this issue

Canadian Medical Association Journal: 189 (30)
CMAJ
Vol. 189, Issue 30
31 Jul 2017
  • Table of Contents
  • Index by author

Article tools

Respond to this article
Print
Download PDF
Article Alerts
To sign up for email alerts or to access your current email alerts, enter your email address below:
Email Article

Thank you for your interest in spreading the word on CMAJ.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Validation of quality indicators for end-of-life communication: results of a multicentre survey
(Your Name) has sent you a message from CMAJ
(Your Name) thought you would like to see the CMAJ web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
Validation of quality indicators for end-of-life communication: results of a multicentre survey
Daren K. Heyland, Peter Dodek, John J. You, Tasnim Sinuff, Tim Hiebert, Carolyn Tayler, Xuran Jiang, Jessica Simon, James Downar
CMAJ Jul 2017, 189 (30) E980-E989; DOI: 10.1503/cmaj.160515

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
‍ Request Permissions
Share
Validation of quality indicators for end-of-life communication: results of a multicentre survey
Daren K. Heyland, Peter Dodek, John J. You, Tasnim Sinuff, Tim Hiebert, Carolyn Tayler, Xuran Jiang, Jessica Simon, James Downar
CMAJ Jul 2017, 189 (30) E980-E989; DOI: 10.1503/cmaj.160515
Digg logo Reddit logo Twitter logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like

Jump to section

  • Article
    • Abstract
    • Methods
    • Results
    • Interpretation
    • Footnotes
    • References
  • Figures & Tables
  • Related Content
  • Responses
  • Metrics
  • PDF

Related Articles

  • Measuring quality of end-of-life communication and decision-making: Do we have this right?
  • PubMed
  • Google Scholar

Cited By...

  • Using Natural Language Processing to Classify Serious Illness Communication with Oncology Patients
  • Quality of end-of-life communication in 2 high-risk ICU cohorts: a retrospective cohort study
  • Clinical Utility of Automatable Prediction Models for Improving Palliative and End-Of-Life Care Outcomes: Towards Routine Decision Analysis Before Implementation
  • Identification and operationalisation of indicators to monitor successful uptake of advance care planning policies: a modified Delphi study
  • Can a patient-directed video improve inpatient advance care planning? A prospective pre-post cohort study
  • Improving hospital-based communication and decision-making about scope of treatment using a standard documentation tool
  • Measuring quality of end-of-life communication and decision-making: Do we have this right?
  • Google Scholar

More in this TOC Section

  • The risk of death or unplanned readmission after discharge from a COVID-19 hospitalization in Alberta and Ontario
  • The link between medical conditions and fatal drownings in Canada: a 10-year cross-sectional analysis
  • Projected estimates of cancer in Canada in 2022
Show more Research

Similar Articles

 

View Latest Classified Ads

Content

  • Current issue
  • Past issues
  • Collections
  • Sections
  • Blog
  • Podcasts
  • Alerts
  • RSS
  • Early releases

Information for

  • Advertisers
  • Authors
  • Reviewers
  • CMA Members
  • Media
  • Reprint requests
  • Subscribers

About

  • General Information
  • Journal staff
  • Editorial Board
  • Advisory Panels
  • Governance Council
  • Journal Oversight
  • Careers
  • Contact
  • Copyright and Permissions
  • Accessibiity
  • CMA Civility Standards
CMAJ Group

Copyright 2022, CMA Impact Inc. or its licensors. All rights reserved. ISSN 1488-2329 (e) 0820-3946 (p)

All editorial matter in CMAJ represents the opinions of the authors and not necessarily those of the Canadian Medical Association or its subsidiaries.

To receive any of these resources in an accessible format, please contact us at CMAJ Group, 500-1410 Blair Towers Place, Ottawa ON, K1J 9B9; p: 1-888-855-2555; e: cmajgroup@cmaj.ca

Powered by HighWire