Promoting effective guideline use in Ontario

The Ministry of Health and Long-Term Care and the Ontario Medical Association (OMA) negotiate 3-year agreements for funding medical services in Ontario. The quality of care provided to the population of Ontario and accountability regarding the utilization of services paid for by the system is of interest to both parties. During the 1997 negotiations, it was agreed that a committee with 3 representatives from the ministry, 3 from the OMA and 1 ex-officio member of the Institute for Clinical Evaluative Sciences should be formed to promote the adoption of evidence-based clinical practice guidelines in the province and to consult widely with the profession in the process. The Guideline Advisory Committee (GAC) was thus established; it reports to the Physician Services Committee, another joint initiative resulting from the negotiations.

In this article we describe the methods that have been developed over the last 4 years to identify well-developed guidelines and some of the strategies being proposed for their dissemination, implementation and evaluation.

Clinical practice guidelines are “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances.”1 Practice guidelines have become widely available through Internet technology. Most guidelines are produced by specific interest groups and are disseminated by publication in a medical journal or by traditional mail; however, seldom have guidelines had any noticeable impact on clinical practice.2 The large volume of guidelines creates confusion for practitioners, who often follow none of them because of the time required to assess the quality of each.3 The GAC thus determined that its first priority was to identify the highest quality guidelines available on selected topics and then to promote their dissemination across the province.

In producing a list of priority topics for guideline assessment, the committee took the following factors into account: feedback from the OMA sections indicated that there was considerable confusion for practitioners over conflicting advice for appropriate practice; utilization data from the ministry demonstrated that the use of numerous procedures had increased rapidly over previous years; and feedback from practising physicians identified areas in which they felt that there was a need for guidelines to aid practice.

Literature searches were then conducted by librarians at the University of Toronto to find all English-language guidelines published in the past 10 years on specific topics (10 primary care areas were initially identified, including asthma, otitis media, osteoporosis and depression). Various associations and interest groups in Ontario were contacted to determine whether there were any unpublished guidelines. After attempts to develop its own scoring methodology to assess the quality of guidelines, the GAC decided to adapt the Appraisal Instrument for Clinical Guidelines.4 This instrument has been validated internationally and consists of 37 items addressing 3 dimensions: rigour of development, context and content, and application.

Recognizing that it would be controversial to recommend guidelines on the basis of the quality of the process by which they were produced, the GAC felt that it was important to develop a rigorous and objective process to determine which guidelines should be recommended. Fellows from the University of Toronto Department of Family and Community Medicine and community-based family physician volunteers from the OMA were brought together in 3 workshops. Each workshop, attended by 10–20 participants, involved a half-day session on the objectives of the GAC and how to assess and validate guideline scores. At the end of the session interested participants were provided with an additional 5 guidelines to assess in the subsequent 2 weeks. Their appraisals were evaluated for consistency and interrater reliability. To date, 34 trained assessors are reviewing guidelines. Each guideline is evaluated by 3 independent assessors, and guidelines identified as warranting further review are assessed for clinical relevance. Over 250 published guidelines addressing 25 clinical areas are currently being assessed. Recommended guidelines in the initial 10 clinical areas are available on the GAC Web site (www.gacguidelines.ca).

The GAC anticipates that guidelines found to be of excellent quality but not convenient for use in clinical practice will need to be reformatted into user-friendly summaries. Volunteer physicians from the community will be asked to evaluate such summaries and provide feedback for improvement. Only the most rigorously developed guidelines will be posted on the Web site in the form of structured abstracts, although interested practitioners can obtain the scores of other guidelines reviewed in a particular clinical area.

Continuing medical education (CME) literature on dissemination strategies indicates that a single method, such as mass mailing, has a minimal effect on changing medical practice.3 The GAC is currently considering a number of options to enhance the dissemination of the best available guidelines.

One option is the Ontario Guideline Collaborative, a group of representatives from provincial licensing bodies, government, university and hospital sectors that convenes 3–4 times per year to discuss dissemination and implementation of recommended guidelines. Other provincial initiatives of interest include a community-based intervention strategy developed by the Partners for Appropriate Anti-Infective Community Therapy to disseminate guidelines on the appropriate use of anti-infectives:5 using the train-the-trainer model, opinion leaders from 50 communities across Ontario have been prepared to implement the program in their respective regions. The GAC is considering using this model to disseminate a chosen guideline on 3 or 4 occasions during the year.

Over the past 10 years more than 3000 small groups of physicians have been formed to work together over carefully prepared, evidence-based materials designed to improve practice on a variety of topics. Participants select the topic of most interest to the group and meet to determine how they could modify their practice according to the evidence presented.6 Adapting this format of the practice-based small group CME session offers the GAC a further strategy for guideline dissemination.

The Ontario College of Family Physicians has developed a train-the-trainer method of facilitating small groups adapting new practice strategies.7 Identifying evidence-based guidelines and modifying them to fit the template produced for this program is yet another potential dissemination strategy for the GAC.

From the current CME literature, we believe that the combination of initiatives in Ontario has the potential to improve clinical practice in the province. The GAC hopes to use provincial data sets to measure the impact of particular guideline dissemination strategies on physician practices. We look forward to reporting on the results of these projects as they unfold.