Skip to main content

Main menu

  • Home
  • COVID-19
    • Articles & podcasts
    • Blog posts
    • Collection
    • News
  • Content
    • Current issue
    • Past issues
    • Early releases
    • Collections
    • Sections
    • Blog
    • Infographics & illustrations
    • Podcasts
    • Classified ads
  • Authors
    • Overview for authors
    • Submission guidelines
    • Submit a manuscript
    • Forms
    • Editorial process
    • Editorial policies
    • Peer review process
    • Publication fees
    • Reprint requests
  • CMA Members
    • Overview for members
    • Earn CPD Credits
    • Print copies of CMAJ
    • Career Ad Discount
  • Subscribers
    • General information
    • View prices
  • Alerts
    • Email alerts
    • RSS
  • JAMC
    • À propos
    • Numéro en cours
    • Archives
    • Sections
    • Abonnement
    • Alertes
  • CMAJ JOURNALS
    • CMAJ Open
    • CJS
    • JAMC
    • JPN

User menu

Search

  • Advanced search
CMAJ
  • CMAJ JOURNALS
    • CMAJ Open
    • CJS
    • JAMC
    • JPN
CMAJ

Advanced Search

  • Home
  • COVID-19
    • Articles & podcasts
    • Blog posts
    • Collection
    • News
  • Content
    • Current issue
    • Past issues
    • Early releases
    • Collections
    • Sections
    • Blog
    • Infographics & illustrations
    • Podcasts
    • Classified ads
  • Authors
    • Overview for authors
    • Submission guidelines
    • Submit a manuscript
    • Forms
    • Editorial process
    • Editorial policies
    • Peer review process
    • Publication fees
    • Reprint requests
  • CMA Members
    • Overview for members
    • Earn CPD Credits
    • Print copies of CMAJ
    • Career Ad Discount
  • Subscribers
    • General information
    • View prices
  • Alerts
    • Email alerts
    • RSS
  • JAMC
    • À propos
    • Numéro en cours
    • Archives
    • Sections
    • Abonnement
    • Alertes
  • Visit CMAJ on Facebook
  • Follow CMAJ on Twitter
  • Follow CMAJ on Pinterest
  • Follow CMAJ on Youtube
  • Follow CMAJ on Instagram
Commentary

Toward evidence-based policy

Tara Kiran
CMAJ October 18, 2016 188 (15) 1065-1066; DOI: https://doi.org/10.1503/cmaj.160692
Tara Kiran
Department of Family and Community Medicine, St. Michael’s Hospital, University of Toronto, and Centre for Urban Health Solutions, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Toronto, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: tara.kiran@utoronto.ca
  • Article
  • Responses
  • Metrics
  • PDF
Loading

See also page E375 and www.cmaj.ca/lookup/doi/10.1503/cmaj.150858 and page 1063 and www.cmaj.ca/lookup/doi/10.1503/cmaj.161084

Many people have become enamoured with the idea that targeted payments to physicians can improve quality of care. Real-life experiences, however, have been almost universally disappointing. At first glance, a linked study by Lavergne and colleagues1 seems to be yet another example of a failed attempt at “pay for performance” in health care. But, on closer examination, the study’s findings are less an indictment of financial incentives and more an example of governments’ failing to take an evidence-based approach to policy-making.

Lavergne and colleagues undertook a rigorous post-hoc evaluation of the annual complex care management fee, one of several new primary care fees introduced in British Columbia over the last decade. British Columbia has maintained a traditional fee-for-service payment system for primary care physicians, and many of the new fee codes were intended to encourage different ways of working to meet the needs of patients with chronic conditions. The complex care management fee was set at $315 per patient per year and ultimately accounted for 3% of total billings for participating primary care physicians. Physicians could bill the fee if they completed a complex care plan for a patient with two or more eligible conditions, such as congestive heart failure and diabetes. The study’s authors found that introduction of the fee had no significant impact on the number of primary care contacts or continuity with the designated primary care physician, that there was no associated decrease in all-cause hospital admissions and that cost per patient rose.

Post-hoc evaluations of policy, however, are always tricky because the objectives of policy-makers are often unclear, the most relevant clinical data are usually unavailable and it is impossible to cleanly disentangle the effect of the policy from other factors influencing outcomes. Perhaps it is unsurprising then that this study leaves us with more questions than answers.

First, we do not know whether the new fee code changed physicians’ approach to management of complex chronic disease or whether some physicians just got paid more for what they were already doing. Lavergne and colleagues found that about two-thirds of eligible patients had the fee code billed during the study period. But, these patients were more likely to have higher continuity, fewer hospital admissions and lower costs than patients who did not have a physician bill the incentive, and that was true even before billing of the incentive. These patients may have been fortunate to have primary care physicians who engaged in complex care planning before introduction of the incentive. Observational evaluations from other jurisdictions have found that pay for performance rewards physicians with higher baseline performance2,3 and that payment reforms can favour physicians who treat healthier patients.4

Second, we do not know how physicians who billed the fee code implemented the recommendations on care plans. There are parallels with efforts around medication reconciliation. Despite imperfect evidence, medication reconciliation has been included in accreditation standards in the United States and Canada. Critics hypothesize that whereas some organizations invested in the resources required for successful implementation, others have simply improved documentation of medication reconciliation.5 Similarly, it is unclear whether physicians who billed the fee code embedded care plans into day-to-day management or whether the focus was on improved documentation, possibly for activities they felt they were already doing.

Third, it is uncertain whether even ideal implementation of care plans would reduce hospital visits. Comprehensive care plans seem like common sense and have been encouraged by government agencies in other jurisdictions.6,7 But despite the propagation of care plans, there is little evidence linking them to better outcomes. There is evidence supporting care coordination for patients with more than one chronic condition, but successful interventions involve clear roles for other health professionals, leverage health information technology and have largely been implemented in high-performing primary care practices.8,9

Finally, it is unclear what the BC government hoped to achieve with introduction of the fee code. The authors of the linked study hypothesized that the incentives were primarily intended to reduce health care use and costs. They are likely right. But perhaps the government was aiming for an improved care experience for patients and families, or greater patient self-confidence in disease management. Maybe they were trying to get physicians to accept more patients with complex illness, or address relative pay differences between family physicians and specialists. If the latter was the goal, then the policy was likely a success.

Governments should be explicit about their goals for reform and incorporate evidence into policy-making. This type of approach can be challenging. Evidence to guide change is not always available, rigorous evaluation can be difficult and time-consuming, and clearly defined goals increase the risk of overt failure.

At the time BC introduced the new fee code, policy experts touted the potential benefits of pay for performance, and the United Kingdom had just introduced one of the most ambitious primary care pay-for-performance schemes in the world. But, there was little evidence supporting the approach. Even now, the most recent systematic review on the topic found there was little evidence to support or not support financial incentives to improve quality in primary care.10 Notably, the systematic review found only seven high-quality studies on the topic — despite widespread attempts at physician-payment reform worldwide.

Lack of evidence makes it all the more important for governments to clarify goals and prospectively evaluate payment reform. Physicians and the public would need to know that the intervention might not result in the desired outcome. Ideally there would be a shared understanding at the outset to halt an intervention that did not meet prespecified objectives. A retrospective analysis from the UK found that withdrawing incentives did not reduce quality of care.11

British Columbia spent $240 million dollars on the Complex Care Initiative over six years. Could this money have been spent on evidence-based interventions that are currently underprovided, such as psychotherapy for patients with mental illness? Other Canadian provinces have also spent large sums on physician-payment reform and have come under criticism for not evaluating the impact of changes.12 Payment reform can be challenging because it is negotiated between governments and medical associations. But, in an era of increasing accountability and fiscal restraint, both parties should be transparent about the health system goals underpinning reforms, how reforms will be evaluated and what steps would be taken if goals are not achieved.

Key points
  • Post-hoc evaluations of policy are tricky because the objectives are often unclear, the most relevant clinical data are usually unavailable and it is impossible to disentangle the effects of the policy from other factors influencing outcomes.

  • There is little high-quality evidence to support or not support financial incentives to improve quality in primary care, despite widespread attempts at physician-payment reform worldwide.

  • Governments should prospectively evaluate payment reforms and commit to change if objectives are not met.

Footnotes

  • Competing interests: None declared.

  • This article was solicited and has not been peer reviewed.

References

  1. ↵
    1. Lavergne R,
    2. Law M,
    3. Peterson S,
    4. et al
    . A population-based analysis of incentive payments to primary care physicians for the care of patients with complex conditions. CMAJ 2016;188:E375–83.
    OpenUrlAbstract/FREE Full Text
  2. ↵
    1. Kiran T,
    2. Victor JC,
    3. Kopp A,
    4. et al
    . The relationship between financial incentives and quality of diabetes care in Ontario, Canada. Diabetes Care 2012;35:1038–46.
    OpenUrlAbstract/FREE Full Text
  3. ↵
    1. Rosenthal MB,
    2. Frank RG,
    3. Li Z,
    4. et al
    . Early experience with pay-for-performance: from concept to practice. JAMA 2005; 294: 1788–93.
    OpenUrlCrossRefPubMed
  4. ↵
    1. Glazier RH,
    2. Klein-Geltink J,
    3. Kopp A,
    4. et al
    . Capitation and enhanced fee-for-service models for primary care reform: a population-based evaluation. CMAJ 2009;180:E72.
    OpenUrlAbstract/FREE Full Text
  5. ↵
    1. Pevnick JM,
    2. Shane R,
    3. Schnipper JL
    . The problem with medication reconciliation. BMJ Qual Saf 2016;25:726–30.
    OpenUrlFREE Full Text
  6. ↵
    1. Edwards ST,
    2. Landon BE
    . Medicare’s chronic care management payment — payment reform for primary care. N Engl J Med 2014;371:2049–51.
    OpenUrlCrossRefPubMed
  7. ↵
    Transforming Ontario’s health care system: Community Health Links provide coordinated, efficient and effective care to patients with complex needs. Toronto: Ontario Ministry of Health and Long-Term Care; modified 2016. Available: www.health.gov.on.ca/en/pro/programs/transformation/community.aspx (accessed 2016 June 14).
  8. ↵
    1. Bodenheimer T,
    2. Ghorob A,
    3. Willard-Grace R,
    4. et al
    . The 10 building blocks of high-performing primary care. Ann Fam Med 2014;12:166–71.
    OpenUrlAbstract/FREE Full Text
  9. ↵
    1. Katon WJ,
    2. Lin EH,
    3. Von Korff M,
    4. et al
    . Collaborative care for patients with depression and chronic illnesses. N Engl J Med 2010;363:2611–20.
    OpenUrlCrossRefPubMed
  10. ↵
    1. Scott A,
    2. Sivey P,
    3. Ait Ouakrim D,
    4. et al
    . The effect of financial incentives on the quality of health care provided by primary care physicians. Cochrane Database Syst Rev 2011;(9): CD008451.
  11. ↵
    1. Kontopantelis E,
    2. Springate D,
    3. Reeves D,
    4. et al
    . Withdrawing performance indicators: retrospective analysis of general practice performance under UK Quality and Outcomes Framework. BMJ 2014;348:g330.
    OpenUrlAbstract/FREE Full Text
  12. ↵
    Chapter 3.06: Funding alternatives for family physicians. In: Annual report of the Office of the Auditor General of Ontario. Toronto: Ministry of Health and Long-Term Care; 2011:150–70.
PreviousNext
Back to top

In this issue

Canadian Medical Association Journal: 188 (15)
CMAJ
Vol. 188, Issue 15
18 Oct 2016
  • Table of Contents
  • Index by author

Article tools

Respond to this article
Print
Download PDF
Article Alerts
To sign up for email alerts or to access your current email alerts, enter your email address below:
Email Article

Thank you for your interest in spreading the word on CMAJ.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Toward evidence-based policy
(Your Name) has sent you a message from CMAJ
(Your Name) thought you would like to see the CMAJ web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
Toward evidence-based policy
Tara Kiran
CMAJ Oct 2016, 188 (15) 1065-1066; DOI: 10.1503/cmaj.160692

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
‍ Request Permissions
Share
Toward evidence-based policy
Tara Kiran
CMAJ Oct 2016, 188 (15) 1065-1066; DOI: 10.1503/cmaj.160692
Digg logo Reddit logo Twitter logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Footnotes
    • References
  • Responses
  • Metrics
  • PDF

Related Articles

  • Realizing the vision of patient-relevant clinical research
  • A population-based analysis of incentive payments to primary care physicians for the care of patients with complex disease
  • Highlights
  • PubMed
  • Google Scholar

Cited By...

  • Realizing the vision of patient-relevant clinical research
  • Google Scholar

More in this TOC Section

  • Transforming race-based health research in Canada
  • Not neutral: reimagining antiracism as a professional competence
  • Time to dismantle systemic anti-Black racism in medicine in Canada
Show more Commentary

Similar Articles

Collections

  • Topics
    • Health policy
    • Health economics
    • Family medicine, general practice, primary care

Content

  • Current issue
  • Past issues
  • Collections
  • Sections
  • Blog
  • Podcasts
  • Alerts
  • RSS
  • Early releases

Information for

  • Advertisers
  • Authors
  • Reviewers
  • CMA Members
  • Media
  • Reprint requests
  • Subscribers

About

  • General Information
  • Journal staff
  • Editorial Board
  • Governance Council
  • Journal Oversight
  • Careers
  • Contact
  • Copyright and Permissions

Copyright 2021, Joule Inc. or its licensors. All rights reserved. ISSN 1488-2329 (e) 0820-3946 (p)

All editorial matter in CMAJ represents the opinions of the authors and not necessarily those of the Canadian Medical Association or its subsidiaries.

Powered by HighWire