Skip to main content

Main menu

  • Home
  • Content
    • Current issue
    • Past issues
    • Early releases
    • Collections
    • Sections
    • Blog
    • Infographics & illustrations
    • Podcasts
    • COVID-19 Articles
    • Obituary notices
  • Authors & Reviewers
    • Overview for authors
    • Submission guidelines
    • Submit a manuscript
    • Forms
    • Editorial process
    • Editorial policies
    • Peer review process
    • Publication fees
    • Reprint requests
    • Open access
    • Patient engagement
  • Members & Subscribers
    • Benefits for CMA Members
    • CPD Credits for Members
    • Subscribe to CMAJ Print
    • Subscription Prices
    • Obituary notices
  • Alerts
    • Email alerts
    • RSS
  • JAMC
    • À propos
    • Numéro en cours
    • Archives
    • Sections
    • Abonnement
    • Alertes
    • Trousse média 2023
    • Avis de décès
  • CMAJ JOURNALS
    • CMAJ Open
    • CJS
    • JAMC
    • JPN

User menu

Search

  • Advanced search
CMAJ
  • CMAJ JOURNALS
    • CMAJ Open
    • CJS
    • JAMC
    • JPN
CMAJ

Advanced Search

  • Home
  • Content
    • Current issue
    • Past issues
    • Early releases
    • Collections
    • Sections
    • Blog
    • Infographics & illustrations
    • Podcasts
    • COVID-19 Articles
    • Obituary notices
  • Authors & Reviewers
    • Overview for authors
    • Submission guidelines
    • Submit a manuscript
    • Forms
    • Editorial process
    • Editorial policies
    • Peer review process
    • Publication fees
    • Reprint requests
    • Open access
    • Patient engagement
  • Members & Subscribers
    • Benefits for CMA Members
    • CPD Credits for Members
    • Subscribe to CMAJ Print
    • Subscription Prices
    • Obituary notices
  • Alerts
    • Email alerts
    • RSS
  • JAMC
    • À propos
    • Numéro en cours
    • Archives
    • Sections
    • Abonnement
    • Alertes
    • Trousse média 2023
    • Avis de décès
  • Visit CMAJ on Facebook
  • Follow CMAJ on Twitter
  • Follow CMAJ on Pinterest
  • Follow CMAJ on Youtube
  • Follow CMAJ on Instagram
Commentary

Further disquiet on the guidelines front

Steven J. Lewis
CMAJ July 24, 2001 165 (2) 180-181;
Steven J. Lewis
Mr. Lewis is Adjunct Professor of Health Policy in the Department of Community Health Sciences, University of Calgary, Calgary, Alta., and President of Access Consulting Ltd., Saskatoon, Sask.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Responses
  • Metrics
  • PDF
Loading

More discouraging news about clinical practice guidelines (CPGs). In this issue (page 157) Ian Graham and colleagues review the quality of drug therapy CPGs developed or endorsed by Canadian organizations from 1994 to 1998.1 Only 14.7% of the 217 guidelines reviewed met half or more of the 20 criteria defining the rigour of the development process, and independent reviewers recommended only 9.2% as being sound without modification. The drug therapy CPGs, it would seem, often retail opinion based on incomplete or haphazardly compiled scientific evidence. For much of the past decade, proponents of evidence-based medicine have decried the lack of uptake of CPGs. Given their quality, as reported by Graham and colleagues, perhaps we should instead applaud.

Or perhaps not. The authors assessed the guidelines using the Appraisal Instrument for Clinical Guidelines, in which 20 of the 37 appraisal criteria assess the rigour of methods used in guideline development. Almost all of the criteria are proxies for, rather than direct indicators of, the quality of the scientific evidence underlying the guidelines. Only 2 deal directly with quality: Are the sources of information used to select the evidence on which the recommendations are based adequate? and Are the methods for rating the evidence satisfactory? (The answer to both questions was No in over 80% of the CPGs that described this information.) However, even a guideline whose process and conclusions are based on good science would not pass muster if this connection were not transparent. Explicitness and transparency are the cornerstones of evidence-based medicine, and it is inexcusable to produce a CPG without coming clean about the methods and bases for recommendations. But it is possible for experts to produce a guideline compatible with the science without systematically reviewing and transparently reproducing the evidence, just as it is possible for a panel to have at hand a definitive, Cochrane-compatible literature review and arrive at unjustifiable conclusions.

Five of the criteria in the appraisal instrument deal with application or implentation of guidelines: for example, Does the guideline document suggest possible methods for dissemination and implementation? and Does the guideline document define measurable outcomes that can be monitored? Predictably, these criteria were universally unmet. CPG developers are no more responsible for the fate of their products than are authors of scientific articles for theirs. Social marketing is optional. At first blush it seems unreasonable to impose these requirements on CPG developers already charged with the difficult and at times fractious task of distilling science into practitioner-friendly recommendations.

But perhaps we should blush twice and upgrade our expectations in line with the appraisal instrument criteria. Our collective response to the evidence that practitioners only rarely follow guidelines, and sometimes think they do when they don't,2 is to produce more guidelines rather than rethink the whole enterprise. I have personally been involved in Western and national exercises (the latter funded by the Federal/Provincial/Territorial Advisory Committee on Health Services) that sought to address the cacophony of duplicated guidelines development by suggesting a common approach to reviewing the literature, populating CPG panels and other measures. Nothing came of either; here, as throughout the health care system, turf battles, parochialism and flagging will impede progress.

We might also remind ourselves to compare CPGs not with perfection but, rather, with current practice. Variations in practice are wide and persistent,3 almost certainly evidence of troublesome quality and accessibility problems. Appropriate use of just about any CPG developed by thoughtful people is likely to improve practice. Almost three-quarters of the guidelines assessed by the appraisers in the study by Graham and colleagues were recommended as is or (in most cases) with modifications despite general inattention to the appraisal instrument criteria. That both “pure” quality and transparency were generally deficient did not mean the guidelines were without clinical merit. Reflexive rejection of almost any guideline is as unwarranted as robotic adherence. Guidelines defensibly rated as unacceptable should be culled from the herd, but until we put in place high-quality, standardized development processes, all CPGs should be judged more on the basis of the wisdom of their recommendations than on the explicit processes of their creation.

Graham and colleagues conclude rather modestly with the suggestion that the quality of all CPGs in Canada be systematically assessed and that the results be disseminated widely and included in CMA Infobase (the CMA's online database of Canadian CPGs). Good ideas. But much more needs to be done. CPGs, even great CPGs, are ignored because there is neither reward for following them nor penalty for ignoring them. Information systems are too feeble to inform practitioners about their absolute and relative performance on most dimensions of practice, let alone adherence to CPGs. It is inevitable that the production of multiple CPGs on the same topic will be confusing, will lead to selective adoption of practices that reflect preferences and prejudices rather than evidence-based judgement and will create yet another excuse to do nothing about practice variations and perverse incentives.

If we are to persist in the CPG business, there should be one Canadian guideline for each area, nationally produced, federally funded and compatible with the criteria contained in the Appraisal Instrument for Clinical Guidelines. If practitioners are going to ignore CPGs, they might as well ignore the best.

Footnotes

  • Competing interests: None declared.

References

  1. 1.↵
    Graham ID, Beardall S, Carter AO, Glennie J, Hébert PC, Tetroe JM, et al. What is the quality of drug therapy clinical practice guidelines in Canada? CMAJ 2001;165(2):157-63. Available: www.cma.ca/cmaj/vol-165/issue-2/0157.asp
    OpenUrlAbstract/FREE Full Text
  2. 2.↵
    Lomas J, Anderson GM, Domnick-Pierre K, Vayda E, Enkin MW, Hannah WJ. Do practice guidelines guide practice? The effect of a consensus statement on the practice of physicians. N Engl J Med 1989;321:1306-11.
    OpenUrlCrossRefPubMed
  3. 3.↵
    Goel V, Williams JI, Anderson GM, Blackstien-Hirsch P, Fooks C, Naylor CD, editors. The ICES practice atlas: patterns of health care in Ontario. 2nd ed. Ottawa: Canadian Medical Association; 1996.
PreviousNext
Back to top

In this issue

CMAJ
Vol. 165, Issue 2
24 Jul 2001
  • Table of Contents
  • Index by author

Article tools

Respond to this article
Print
Download PDF
Article Alerts
To sign up for email alerts or to access your current email alerts, enter your email address below:
Email Article

Thank you for your interest in spreading the word on CMAJ.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Further disquiet on the guidelines front
(Your Name) has sent you a message from CMAJ
(Your Name) thought you would like to see the CMAJ web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
Further disquiet on the guidelines front
Steven J. Lewis
CMAJ Jul 2001, 165 (2) 180-181;

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
‍ Request Permissions
Share
Further disquiet on the guidelines front
Steven J. Lewis
CMAJ Jul 2001, 165 (2) 180-181;
Digg logo Reddit logo Twitter logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like

Jump to section

  • Article
    • Footnotes
    • References
  • Responses
  • Metrics
  • PDF

Related Articles

  • No related articles found.
  • PubMed
  • Google Scholar

Cited By...

  • Guideline adaptation: an approach to enhance efficiency in guideline development and improve utilisation
  • Evaluation and adaptation of clinical practice guidelines
  • Evidence-Based Guidelines in Laboratory Medicine: Principles and Methods
  • Assessing the quality of clinical practice guidelines
  • Time to weed the CPG garden
  • Le moment est venu de desherber le jardin des guides de pratique:
  • Google Scholar

More in this TOC Section

  • Ensuring timely genetic diagnosis in adults
  • The case for improving the detection and treatment of obstructive sleep apnea following stroke
  • Laser devices for vaginal rejuvenation: effectiveness, regulation and marketing
Show more Commentary

Similar Articles

 

View Latest Classified Ads

Content

  • Current issue
  • Past issues
  • Collections
  • Sections
  • Blog
  • Podcasts
  • Alerts
  • RSS
  • Early releases

Information for

  • Advertisers
  • Authors
  • Reviewers
  • CMA Members
  • CPD credits
  • Media
  • Reprint requests
  • Subscribers

About

  • General Information
  • Journal staff
  • Editorial Board
  • Advisory Panels
  • Governance Council
  • Journal Oversight
  • Careers
  • Contact
  • Copyright and Permissions
  • Accessibiity
  • CMA Civility Standards
CMAJ Group

Copyright 2023, CMA Impact Inc. or its licensors. All rights reserved. ISSN 1488-2329 (e) 0820-3946 (p)

All editorial matter in CMAJ represents the opinions of the authors and not necessarily those of the Canadian Medical Association or its subsidiaries.

To receive any of these resources in an accessible format, please contact us at CMAJ Group, 500-1410 Blair Towers Place, Ottawa ON, K1J 9B9; p: 1-888-855-2555; e: cmajgroup@cmaj.ca

Powered by HighWire