Abstract
Background: Inappropriate health care leads to negative patient experiences, poor health outcomes and inefficient use of resources. We aimed to conduct a systematic review of inappropriately used clinical practices in Canada.
Methods: We searched multiple bibliometric databases and grey literature to identify inappropriately used clinical practices in Canada between 2007 and 2021. Two team members independently screened citations, extracted data and assessed methodological quality. Findings were synthesized in 2 categories: diagnostics and therapeutics. We reported ranges of proportions of inappropriate use for all practices. Medians and interquartile ranges (IQRs), based on the percentage of patients not receiving recommended practices (underuse) or receiving practices not recommended (overuse), were calculated. All statistics are at the study summary level.
Results: We included 174 studies, representing 228 clinical practices and 28 900 762 patients. The median proportion of inappropriate care, as assessed in the studies, was 30.0% (IQR 12.0%–56.6%). Underuse (median 43.9%, IQR 23.8%–66.3%) was more frequent than overuse (median 13.6%, IQR 3.2%–30.7%). The most frequently investigated diagnostics were glycated hemoglobin (underused, range 18.0%–85.7%, n = 9) and thyroid-stimulating hormone (overused, range 3.0%–35.1%, n = 5). The most frequently investigated therapeutics were statin medications (underused, range 18.5%–71.0%, n = 6) and potentially inappropriate medications (overused, range 13.5%–97.3%, n = 9).
Interpretation: We have provided a summary of inappropriately used clinical practices in Canadian health care systems. Our findings can be used to support health care professionals and quality agencies to improve patient care and safety in Canada.
As health care systems struggle with sustainability, there is increased recognition that a substantial percentage of the health care received is inappropriate.1 Inappropriate health care occurs when effective clinical practices are underused, ineffective clinical practices are overused or other practices are misused. It can lead to negative patient experiences,2 poor health outcomes3,4 and inefficient use of scarce health care resources.5 In response, there is widespread professional and policy interest in reducing inappropriate health care in Canada and abroad. For example, in 2014, Choosing Wisely Canada,6 a physician-led campaign in partnership with the Canadian Medical Association, was established. This initiative encourages conversations between clinicians and patients about low-value or overused care in efforts to reduce inappropriate care. Choosing Wisely Canada is endorsed across Canada by all provincial and territorial medical associations (https://choosingwiselycanada.org/about/).
Although reducing inappropriate health care is a high priority for health care professionals, agencies and governments in Canada, designing effective initiatives for quality improvement has been a difficult goal to achieve without knowledge of which clinical practices are inappropriately used. This is further challenged because Canada does not have a mandatory and comprehensive national tracking system for quality. The Canadian Institute for Health Information (CIHI) houses multiple Canadian health databases, but it does not collect information on all clinical practices. Therefore, a systematic synthesis is necessary to provide an overview of inappropriate health care in Canada.7 Summaries of inappropriately used clinical practices exist for several countries: United States,8,9 United Kingdom10 and Australia.11 Each of these syntheses found high levels (50% on average) of inappropriately used practices and laid the foundation for several quality improvement initiatives in these countries. We aimed to conduct a systematic review to estimate the nature and amount of inappropriately used clinical practices in Canada.
Methods
Our protocol was published12 and registered with PROSPERO (the international prospective register of systematic reviews): registration no. CRD42018093495. We used the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)13 statement to guide reporting.
Quality of health care is a multidimensional concept. In this review, we defined quality using the framework put forth by the Institute of Medicine,14 which includes 6 domains of quality care: safe, effective, patient-centred, timely, efficient and equitable. We focused our review on 1 of these quality domains (effectiveness) and reported our findings in terms of inappropriateness (overuse, underuse, misuse) of clinical practices.
Data sources and search strategy
Our search strategy (Appendix 1, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.211416/tab-related-content) is reported according to the PRISMA-S guideline.15,16 It was executed by an experienced information specialist (T.R.), after peer review by a second information specialist using the Peer Review of Electronic Search Strategies (PRESS) checklist.17 We searched MEDLINE, EconLit, Science Citation Index Expanded, Arts & Humanities Citation Index, Emerging Sources Citation and Conference Proceedings Citation Index, and Cochrane Library (all databases). Examples of key search terms used for the concept of inappropriate health care are both specific (“unnecessary procedures,” “inappropriate prescribing”) and comprehensive (“comparative effectiveness research,” “delivery of health care,” “quality of health care”). Controlled vocabulary and natural language terms were applied according to the taxonomy of each database for optimal retrieval. We limited our searching to studies published in 2007 onwards; experts in quality improvement across Canada advised us that it takes a minimum of 10 years to notice a trend in data on inappropriate health care, and that studies older than this were unlikely to be useful in determining priorities for future quality improvement activities. We did not apply language limits or study design filters. The grey literature search included targeted, iterative hand searching of 25 government or research organization websites including those of all provincial and territorial ministries of health, provincial health care quality organizations and administrative data facilities, both provincial (e.g., ICES) and national (e.g., CIHI). We conducted 3 consecutive searches, first from Jan. 1, 2007, to May 28, 2018, and again from June 1, 2018, to Sept. 1, 2019. We conducted a retrospective database search (for additional search terms found in the grey literature) from Jan. 1, 2007, to Sept. 1, 2019. We conducted an updated search using the revised database strategy and of the grey literature from Sept. 1, 2019, to July 20, 2020. We also performed citation checking: we evaluated the reference lists of all included studies to identify additional studies not captured by our search strategy.
Study selection
Two team members independently screened the titles and abstracts identified by the electronic and grey literature searches, and resolved discrepancies by discussion. We included all quantitative study designs reporting data on appropriately or inappropriately used clinical practices in Canada. We defined appropriate and inappropriate practices as ones that did and did not conform fully to an evidence-based recommendation, respectively. Inappropriate care included underuse (failure to provide a clinical practice when patient benefits clearly outweighed the risks), overuse (providing a clinical practice when its potential for harm exceeds the possible benefit) and misuse (when an appropriate clinical practice is selected but a preventable complication occurs and as a result the patient does not receive the full potential benefit of the practice).18 All practices undertaken by a health care professional in a Canadian health care setting were eligible. In line with previous reviews of inappropriate health care in other countries,8–11 we relied on the authors’ identifications of “recommended” clinical practices in the included studies. We included only studies that reported on large or diverse populations, such as the entire nation; 1 or several provinces, territories or cities; or multiple centres.
Data extraction
Data were abstracted in duplicate using a standardized, pilottested form in Distiller SR software.19 In studies where only appropriate health care was reported, we extrapolated inappropriate health care by subtracting the proportion of appropriate care from 100%. We were interested in usual or normal use of clinical practices. Therefore, in longitudinal studies, we extracted the last reported time point, whereas, in experimental studies we extracted baseline measurements for trials with baseline data and postintervention control group data in all other trials.
Assessment of methodological quality
Two reviewers independently assessed the methodological quality of all included studies using the following validated tools: Quality Assessment and Validity Tool for Before/After-Cohort Design Studies, 20,21 Quality Assessment and Validity Tool for Cross-sectional Studies,20–23 Cochrane Risk of Bias Tool 2.0,24 Joanna Briggs Institute Checklist for Quasi-Experimental Studies25 and Joanna Briggs Institute Checklist for Case Series Studies.26 Conflicts regarding all assessments in data extraction and methodological quality were resolved through team discussion.
Data synthesis
We classified all practices first by type of inappropriate use (underuse, overuse or misuse) and, second, as diagnostic or therapeutic. We defined diagnostics as tests used in clinical practice to identify with high accuracy the condition or disease in a patient, and thus to provide early and proper treatment.27 Therapeutics referred to treatment and care of a patient for the purpose of either preventing or treating disease, or alleviating pain or injury.28 In line with a previous review of studies of health care services in the US,9 clinical practices that could function as either diagnostics or therapeutics (e.g., endoscopy and angiography) were classified according to their primary function as stated in the included study. Finally, within diagnostics and therapeutics, we grouped similar practices into subcategories that emerged from the data: diagnostics (referrals, assessments, screening, blood tests, imaging and multiple tests) and therapeutics (acute care procedures, biophysical therapy, psychosocial therapy and medications).
Statistical analysis
To describe the amount of practices identified, we reported proportions and ranges of proportions of inappropriate use for each practice. We determined summaries of inappropriate use by calculating medians and interquartile ranges (IQRs), based on the percentage of patients not receiving a recommended practice (underuse) and receiving a practice when not recommended (overuse). First, we calculated a median proportion and IQR for all inappropriately used practices combined. Second, we calculated an overall median proportion and IQR for all underused practices and all overused practices. Third, for both diagnostics and therapeutics, and their subcategories, we calculated median proportions and IQRs overall and by kind of inappropriate practice. We assessed for significant differences between kinds of inappropriate practice using the Mann–Whitney U Median Test in Statistical Package for the Social Sciences (SPSS) Version 27. We also evaluated for trends over time by reviewing the median proportions for all inappropriate care, diagnostics and therapeutics using the median publication year of 2017 as the cut point (2009–2017 and 2018–2020). All statistics reported are at the study summary level.
We conducted a sensitivity analysis to see if the median proportion estimates changed when methodologically weak studies were omitted.
Ethics approval
This study, being a systematic review, did not require ethics approval.
Results
Figure 1 (PRISMA flow diagram) shows article selection. We screened 16 530 titles and abstracts, of which 930 were potentially relevant, and 174 were included in the systematic review. Studies excluded at full text are detailed in Appendix 2, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.211416/tab-related-content. Of the 174 included studies, 66 (37.9%) evaluated diagnostics, 85 (48.9%) evaluated therapeutics and 23 (13.2%) included both.
Flow chart for selection of articles.
The 174 included studies included 28 900 762 patients aged from birth to 108 years. All health sectors and Canadian jurisdictions are represented in the sample (summary in Table 1). Appendix 3 (available at www.cmaj.ca/lookup/doi/10.1503/cmaj.211416/tab-related-content) provides greater detail on the included studies.
Characteristics of the included studies
A detailed assessment of the methodological quality of the included studies is in Appendix 4 (available at www.cmaj.ca/lookup/doi/10.1503/cmaj.211416/tab-related-content). We rated 47 (27.0%) studies as low methodological quality. The most common reasons for lower quality scores were lack of probabilistic sampling in nonpopulation-based studies and lack of reported instrument reliability and validity.
The 174 included studies assessed 228 unique practices. Ninety-four (54.0%) of the studies reported on 144 underused practices (Table 2) and 95 (54.6%) studies reported on 109 overused practices (Table 3); 25 practices were both under- and overused (Table 2 and Table 3). No studies reported misused practices. One hundred twenty (52.6%) of the practices were diagnostic and 108 (47.4%) were therapeutic. Most practices, whether underused or overused, were reported in a single study (n = 174, 68.8%); 42 (16.6%) practices were reported in 3 or more studies and 15 (5.9%) practices were reported in 5 or more studies (Table 2 and Table 3).
Underused clinical practices*
Overused clinical practices
Median proportions and IQRs for overall inappropriate use, underuse and overuse by care category (i.e., diagnostics or therapeutics) and their 10 subcategories are summarized in Table 4. We found that the median proportion of inappropriate use across all practices was 30.0% (IQR 12.0%–56.6%). Proportions of underuse were statistically higher than proportions of overuse for both diagnostic and therapeutic practices. Variance (indicated by the width of the IQR) was also consistently higher for underuse than for overuse.
Inappropriately used clinical practices
Several evidence sources for assessing the appropriateness or inappropriateness of the 228 clinical practices were reported. Most studies (n = 165, 94.3%) cited a national or international guideline. Other evidence sources included systematic reviews or meta-analyses (n = 29, 16.7%) and quality indicators (n = 1, 0.6%). The evidence sources used in each study are listed in Appendix 3.
We found that 120 unique diagnostic practices were investigated in 89 studies; 78 (65.0%) diagnostic practices were underused (Table 2), 56 (46.7%) were overused (Table 3) and 14 (11.7%) were both underused and overused. Diagnostics were inappropriately used, on average, 28% of the time (IQR 12.7%–50.4%). The lowest overall proportion of inappropriate use of diagnostics was in imaging tests (median 13.8%, IQR 4.5%–29.0%), whereas the highest proportions were in laboratory tests (median 48.4%, IQR 26.4%–73.0%). The most frequently investigated underused diagnostics were glycated hemoglobin (blood tests), lipid tests (blood tests) and diabetic eye examinations (assessments). Glycated hemoglobin, assessed in 9 studies, had underuse proportions of 18.0%–85.7%. Lipid tests, assessed in 8 studies, had underuse proportions of 3.2%–47.0%. Diabetic eye examinations, also assessed in 8 studies, had underuse proportions of 22.9%–80.5%. The most frequently investigated overused diagnostic was thyroid-stimulating hormone (blood tests), investigated in 5 studies with overuse proportions ranging from 3.0%–35.1%. The next most frequently investigated overused diagnostics, evaluated in 4 studies each, were radiography of the chest (imaging; overused 2.4%–34.0%), Papanicolaou test (screening; overused 8.0%–15.7%) and transthoracic echocardiogram (imaging; overused 2.9%–13.8%).
We found that 108 therapeutic practices were investigated in 108 studies: 66 (61.1%) therapeutics were underused (Table 2), 53 (49.1%) were overused (Table 3) and 11 (10.2%) were both underused and overused. Therapeutics were inappropriately used, on average, 34.0% of the time (IQR 10.0%–61.1%), with the lowest overall proportions of inappropriate use for medications (median 25.9%, IQR 5.8%–60.2%) and the highest proportions for acute care procedures (median 53.5%, IQR 21.8%–71.4%). Although acute care procedures (e.g., carotid endarterectomy) had the highest median proportion of inappropriate use, they were among the least investigated therapeutics (10 procedures in 12 studies). The most frequently investigated therapeutics that were underused were statins (medications), with underuse proportions of 18.5%–71.0% (n = 6), and combinations of cardiovascular drugs (medications), with underuse proportions of 3.3%–98.8% (n = 5). The most frequently investigated overused therapeutics were also all within the medication subcategory: overuse ranged from 11.8% to 76.0% for antimicrobials (n = 8), 5.6%–76.5% for antipsychotics (n = 6) and 0.1%–23.9% for opioids (n = 5).
Table 5 displays the medians for inappropriate use over the 12 years of data included in this review, for which we used the median publication year of 2017 as the comparison point (Table 5). The largest difference was in therapeutics, which showed a decrease of 17.7% in inappropriate care in recent years. When assessed by subcategory, only medications showed a noteworthy reduction in inappropriate care (41.0% down to 14.0%). This reduction was due to fewer medications being overused (38% down to 5.0%); underuse of medications increased during the same time frame (46.0% up to 63.0%) (Appendix 5, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.211416/tab-related-content).
Trends in amount of inappropriately used practices over time
When we omitted studies that were methodologically weak, median proportion estimates were largely unchanged (Appendix 6, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.211416/tab-related-content).
Interpretation
We identified 174 studies that investigated 228 unique clinical practices that were underused or overused in Canada over the last decade. The dominant finding from our review is that there are large gaps between the care people should receive and the care they do receive. We found that, on average, 30.0% of the care received by people in Canada as assessed in the included research papers using the Institute of Medicine’s definitions of underuse and overuse,18 was deemed inappropriate. This was true for both diagnostic and therapeutic practices across different health sectors, all age groups, and whether the nation or select cities, provinces or territories were evaluated.
Estimates of the amount of inappropriate care from our review are similar to those reported in reviews from other countries. In the germinal 1998 review of inappropriate health care in the US,8 patients received inappropriate care in 45% of encounters. Like our findings, there was substantial heterogeneity in inappropriate use in the US review based on the clinical practices evaluated, ranging from 21.3% to 89.5%. Similar findings were reported in reviews from other countries: in the UK,10 51%–97% of care was reported to be inappropriate, and in Australia,11 10.0%–87.0% was inappropriate.
Inappropriate care is a pressing problem in health care, largely because it causes iatrogenic harm to patients and often interferes with the delivery of high-value care.204 It also leads to negative patient experiences,2 poor health outcomes3,4 and inefficient use of scarce health care resources.5 Previous reviews8–11 on inappropriate care provided much needed stimuli to the field of health care quality by elevating global recognition that inappropriate care is not only a serious and widespread problem, but one to which no health sector is immune. These reviews also laid the foundation for several successful quality improvement initiatives in their countries (e.g., the 100 000 Lives and Protecting 5 Million Lives from Harm campaigns in the US).18,19 The findings from our review provide examples of clinical practices that are underused and overused in Canada. Knowledge of these indicators is necessary to underpin initiatives in Canada to improve the quality of health care. Our results can be used by provincial and territorial governments and quality improvement organizations to prioritize future quality improvement initiatives. Our findings also provide a critically needed benchmark tool against which future progress in quality improvement can be measured.
Proportions of inappropriate use of many of the clinical practices identified in our review varied widely; however, some practices were studied frequently and others infrequently. As a result, large gaps in our knowledge of inappropriately used clinical practices in Canada remains. Although we were able to provide a substantial listing and summary of inappropriately used practices in Canada, it is not a comprehensive summary of all practices delivered in the Canadian health care system. Additional research, especially on practices not yet investigated and on those less frequently investigated, are critical next steps to expand the list of inappropriately used practices.
Limitations
Studies were heterogeneous with respect to the practices investigated, populations used, data collection time points and how inappropriate care was measured. Although we retrieved and evaluated each cited practice recommendation, it was not feasible to assess the quality of the evidence behind each recommendation. There may be valid reasons not reported in the included studies for why some patients did or did not receive a recommended practice. Our review was limited to studies that evaluated practices against a criterion standard such as a guideline recommendation; this may have led to some missed reports on inappropriate care. We only captured instances of appropriate or inappropriate care that were studied and, thus, where researchers speculated that there was a problem of appropriateness. Many of the included practices were evaluated in a single study, which limited the conclusions that could be drawn on these practices. Finally, a common reason for lower quality scores (i.e., lack of probabilistic sampling in non-population-based studies) may have affected the reliability of some of the inappropriate proportions that we reported.
Conclusion
We found that many clinical practices received by people in Canada are inappropriate; whether that practice is diagnostic or therapeutic, it frequently does not meet recommended standards. Although we identified a considerable range of clinical practices that are inappropriate, it is not an exhaustive listing of all practices delivered in Canada. Further research is necessary to expand on this list. Clinicians and organizations could use the list of clinical practices from this review (especially the 42 most-studied practices) to identify priorities for their work on quality improvement.
Acknowledgement
The authors would like to thank Caitlyn Ford (Manager, Research Information Services, Canadian Agency For Drugs And Technologies In Health) for PRESS reviewing our search strategy.
Footnotes
Competing interests: Alan Katz is a member of the ICES Scientific Advisory Committee and the SAIL International Advisory Board. He is president of the Canadian Association for Health Services and Policy Research and a member of the Board of Directors of The College of Family Physicians Canada. No other competing interests were declared.
This article has been peer reviewed.
Contributors: Janet Squires and Jeremy Grimshaw conceived the study. Tamara Rader developed and ran the search strategy. Janet Squires, Danielle Cho-Young, Laura Aloisio, Simon Decary, Melissa Demery Varin, Megan Greenough, Letitia Nadalin-Penno and Wilmer Santos contributed to screening, data extraction or quality assessment. Janet Squires, Danielle Cho-Young, Laura Aloisio and Jeremy Grimshaw completed the synthesis with input and critical revision from all authors. Janet Squires drafted the manuscript. All of the authors contributed to development of the study design, reviewed and revised the manuscript critically for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.
Funding: This research was supported by a Canadian Institutes of Health Research (CIHR) Project Grant. The CIHR had no role in the study’s design, conduct and reporting. Janet Squires holds a University of Ottawa Research Chair in Health Evidence Implementation. Carole Estabrooks holds a Canada Research Chair in Knowledge Translation (care of older adults). John Lavis holds a Canada Research Chair in Evidence-Informed Health Systems. Jeremy Grimshaw holds a Canada Research Chair in Health Knowledge Transfer and Uptake. Michael Hillmer has received research grants from CIHR and the Public Health Agency of Canada. Alan Katz has received research grants from CIHR.
Data sharing: The search strategy, extracted data, and quality assessment are in the appendices. Citations for all included studies are in manuscript references. Other data sets from this study are available upon reasonable request from the corresponding author.
- Accepted December 16, 2021.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons.org/licenses/by-nc-nd/4.0/
References
In this issue
Article tools
Jump to section
Related Articles
Cited By...
- No citing articles found.
More in this TOC Section
Similar Articles
Collections
