Abstract
Background: Antibiotics are a medication class for which inappropriate prescribing is frequently described. We sought to assess the effectiveness of a mailed intervention combining confidential prescribing feedback with targeted educational bulletins in increasing the use of less expensive, first-line antibiotics by practising physicians.
Methods: The participants were 251 randomly selected primary care physicians from southern Ontario who consented to participate (135 in the feedback group and 116 in the control group). Prescribing data were obtained from the claims database of the Ontario Drug Benefit program, which covers all Ontarians over age 65 years for drugs selected from a minimally restrictive formulary. Confidentially prepared profiles of antibiotic prescriptions coupled with guidelines-based educational bulletins were mailed to the intervention group every 2 months for 6 months. The control group received no intervention until after completion of the study. The main outcome measures were change from baseline in physician's median antibiotic cost and proportion of episodes of care in which a prespecified first-line antibiotic was used first.
Results: The median prescription cost of about $11 remained constant in the feedback group but rose in the control group (change of $0.05 v. $3.37, p < 0.002). First-line drug use increased in the feedback group but decreased in the control group (change of 2.6% v. -1.7%, p < 0.01). In a mailed survey of 100 feedback recipients (response rate 76%), 82% indicated that they would participate readily in another, similar program.
Interpretation: A simple program of confidential feedback and educational materials blunted cost increases, increased the use of first-line antibiotics and was highly acceptable to Ontario primary care physicians.
Modifying prescribing patterns has proved a formidable challenge to those seeking to promote rational drug use. For many medications, neither the publication of primary research nor the accumulation of relevant evidence in a published practice guideline has achieved the desired effect. [1, 2] Strategies to actively promote more evidence-based prescribing [3–11] range from interventions such as academic detailing [6] (office visits by pharmacist-educators) and multidisciplinary approaches, [7, 8] which are effective but labour intensive and may be prohibitively expensive, to mailed promotional campaigns, which have minimal effect on practice. [10, 11] In addition, guideline implementation may be hampered by low acceptability to practitioners, many of whom regard guidelines as unwarranted interference in their practice.
Antibiotics are a medication class for which inappropriate prescribing is frequently described [12] and for which misprescribing may lead to suboptimal care, unnecessary expenditures and the promotion of resistant bacterial strains. In 1994 a provincially funded panel produced the Ontario Anti-infective Guidelines for Community-Acquired Infections. [13] These guidelines listed several first-line and second-line anti-infective choices for 56 common infectious conditions and also gave dosing and cost information.
We wished to evaluate the effectiveness of a simple mailed intervention as a means of promoting implementation of these guidelines. An administrative data set containing comprehensive prescribing information for Ontario residents over the age of 65 years was used to generate the feedback profiles. For outpatients in this age group, the infections most commonly treated are those of the respiratory and urinary tracts. [14] We hypothesized that the feedback and education intervention would lead to an increase in the use of drugs recommended as first-line agents for these conditions. Since none of the new and expensive antibiotics are recommended as first-line choices for these infections, we hypothesized that the feedback intervention would lead to less expensive prescribing. Finally, we hypothesized that, owing to self-selection, at baseline those who consented to receive feedback would use a higher proportion of first-line antibiotics than those who declined.
Methods
Physician subjects
We obtained the names, addresses and prescriber identification numbers of 1095 randomly selected primary care physicians in southern Ontario by linkage of administrative databases. To reduce the chance of contamination between study arms or reinforcement of the intervention through participant interaction, physicians with the same address as another participant were not selected. Baseline prescribing patterns were evaluated, and physicians with fewer than 10 antibiotic claims in a 2-month period were eliminated. Of the 833 remaining eligible physicians, 400 were randomly assigned to the intervention arm and 400 to the control arm. Randomization was carried out before consent, following Zelen's method, [15] since it was important that control physicians not know that their prescribing patterns were being compared to those of physicians receiving an educational intervention.
Physicians randomly assigned to the intervention were invited to consent to a program of prescribing feedback and education around antibiotic use. Ethics approval for the study was received retrospectively from the Research Ethics Board of Sunnybrook Health Science Centre. The physicians were assured that a strict confidentiality protocol was in place whereby no one on the project staff would have access to both their name and their profile and that profiles would not be made available to the Ministry of Health or any outside agency. It was reinforced that no judgement was being made about the appropriateness of the individual prescriptions since claims data do not include sufficient clinical information to make such an evaluation. Initial invitations to participate were followed by 2 reminders to nonresponders at 2-week intervals. Five packages were returned as undeliverable; responses were received from 203 of the 395 physicians who received packages, of whom 135 wished to participate, giving an overall consent rate of 34%.
Because of the possibility that obtaining consent from the participants in the intervention arm would select physicians with more appropriate prescribing patterns, we also obtained consent from the participants in the control group. These physicians were offered confidential prescribing profiles on 1 occasion (delayed until completion of feedback to the intervention group). Ten packages were returned as undeliverable; responses were received from 194 of the 390 physicians who received packages, of whom 116 (30% of those who received invitations) wished to participate.
Intervention
The intervention consisted of mailed packages of prescribing feedback and guidelines-based educational materials, which were sent every 2 months for 6 months. The initial feedback report, mailed Nov. 1, 1996, gave baseline data for November and December 1995. Data for November and December 1996 were sent in mid-January 1997, and data for January and February 1997 in mid-March 1997. Feedback was presented as a bar graph showing the absolute number of antibiotic scripts for the participant and the average for a peer group (the 400 physicians invited to the control arm of the study). Antibiotic data were grouped in 5 categories: first-line drugs (penicillins, macrolides, trimethoprim-sulfamethoxazole, tetracycline and first-generation cephalosporins), ciprofloxacin, norfloxacin, cefaclor and others. The average drug cost for the participant and the peer group defined above was also shown (Appendix 1 Figure 2).
Figure 2. Appendix 1: Sample bar graph showing the average cost of antibiotics prescribed by a physician in the intervention arm of the study in October and November 1995 for Ontario Drug Benefit program recipients (over 65 years of age).
The educational bulletins that accompanied the profiles were written in a brief, informal style with an emphasis on practical tips. They were relatively simply designed in an effort to distinguish them from commercial promotional materials. Topics were selected on the basis of focus group work with primary care physicians regarding barriers to appropriate antibiotic use.
Data source
The profiles were prepared from claims data for prescriptions under the Ontario Drug Benefit program. Available data fields include date of filling, drug identification number, quantity, professional fee, charges claimed from the Ontario Drug Benefit program, and scrambled patient and physician identification numbers whereby the drug history of an individual patient or prescriber could be assembled over time but could not be linked to that person. For study physicians, claims for all antibiotics were extracted from the data set. All unique patient identifiers shown on these claims were then linked back to the main data set to extract any claims for antibiotics in which the prescription was written by a nonstudy physician.
Participants' satisfaction with the intervention was evaluated through a mailed survey (a convenience sample of 100 participants) and a focus group (8 participants).
Analysis
The intervention was offered over the winter months (November 1996 through April 1997). Because of seasonality in antibiotic prescribing patterns, the same months in the previous year were taken as the baseline for comparison purposes. To provide stable estimates of the endpoints, the data for physicians with claims for fewer than 10 prescriptions in the intervention period were omitted from the analysis.
Drug cost
Drug costs within an individual physician's set of claims were nonnormally distributed, with a majority of lower cost first-line agents prescribed (drug cost less than $15) and a smaller number of high-cost or large-volume prescriptions written (drug cost more than $50). Accordingly, we determined median antibiotic costs for each of the participants in the 2 groups. The median costs were relatively normally distributed across physicians in the 2 groups and were compared between groups by means of analysis of variance.
Progressive changes in drug costs across the intervention period were examined by measuring costs for the 2 groups in 2-month windows beginning immediately before the intervention and continuing through the intervention period. We compared changes in antibiotic costs for intervention and control physicians using repeated-measures analysis of variance.
Antibiotic selection
To analyse the choice of antibiotic, we divided drug use into "episodes of care," defined as periods of antibiotic treatment preceded and followed by antibiotic-free periods of at least 21 days. Only the first antibiotic used in an episode of care was considered in analyses of antibiotic choice. For instance, in a case in which a patient had received tetracycline, failed to respond and 5 days later was given ciprofloxacin, we assumed that the ciprofloxacin was not being used in a first-line role and included only the tetracycline in the drug selection analysis.
To assess whether the consent process selected a nonrepresentative group of prescribers, we compared prescribing patterns for all participants and all nonparticipants at baseline using a t-test.
Results
Physicians in the intervention and control arms of the study were similar with regard to age, sex, number of years since graduation from medical school and certification by the College of Family Physicians (Table 1).
Table 1. Characteristics of Ontario physicians participating in a program of prescribing feedback and education around antibiotic use
Drug costs
Drug costs were evaluated for 134 physicians in the intervention group and 116 physicians in the control group; 1 intervention physician had fewer than 10 prescriptions in the study period, and the data for this doctor were not included. Median antibiotic costs from 1995/96 to 1996/97 were unchanged in the intervention group but increased by an average of over $3 per prescription in the control group (p < 0.002) (Table 2). For comparison purposes, the same analysis was applied to nonsteroidal anti-inflammatory drugs, a drug class for which no education or feedback was provided. There was no difference between the 2 groups in the costs of these agents over the same period. Changes in the participants' median antibiotic costs over the course of the intervention are shown in Figure 1. For this analysis, the 106 intervention physicians and 94 control physicians who had at least 10 prescriptions in each 2-month period were included. Repeated-measures analysis of variance showed a significant difference in antibiotic cost over time between the 2 groups (p < 0.01).
Table 2. Antibiotic prescribing patterns in 1995/96 (baseline) and 1996/97 (intervention)
Figure 1. Effect of prescribing feedback and education program on antibiotic cost. Intervention began in November 1996. Costs are shown as the group mean of individual physicians' median antibiotic cost (ingredient cost only).
Antibiotic selection
In most episodes of care (87.1%), only a single antibiotic was used. Among the 12 447 episodes in which a first-line drug was used first, a second antibiotic was prescribed in 11.9%. Among the 5788 episodes of care in which an antibiotic other than a first-line drug was prescribed first, a second antibiotic was used in 15.0%. The proportion of episodes of care in which a first-line antibiotic was used first rose in the intervention group but fell in the control group over the intervention period (change of 2.6% v. -1.7%, p < 0.01) (Table 2).
Volunteer bias
The prescribing patterns of the nonparticipants were compared to those of all participants before the intervention. The participants used a higher proportion of first-line agents than the nonparticipants (64.2% v. 60.7%, p < 0.02). This pattern of antibiotic selection was also reflected in the median antibiotic cost, which was lower for the participants ($12.54 vs. $15.70, p < 0.001).
Program evaluation
After a single mailing, 76 of 100 mailed questionnaires were completed and returned. A total of 82% of the respondents indicated that they would participate readily in another, similar program. Factors that they identified as "very important" in making a program of this type acceptable to physicians included protection of their confidentiality and assurance that the data be prepared at arm's length from the Ministry of Health (Table 3). The focus group participants endorsed the program as relevant, useful and enjoyable. They reported the peer comparison to be one of the most interesting components of the program. Despite instructions to the contrary, they saw the peer group profile as the target toward which they should be aiming.
Table 3. Results of participant evaluation of the program (mailed survey, n = 76)
Interpretation
Our results show that a simple intervention combining confidential prescribing feedback and targeted educational materials can blunt cost increases, promote selection of first-line antibiotics and be favourably received by physicians. The feedback strategy is attractive because it is inexpensive and can be implemented readily even in geographically remote regions, where conventional continuing medical education interventions may be less accessible. Potential cost savings from a program with the effect size observed in this study are substantial given that outpatient antibiotic expenditures in Ontario exceed $30 million per year [16] in the public sector alone. The rather modest change in the proportion of first-line agents prescribed, which appears discordant with the marked change in prescription costs, is likely due to imprecision in the definition of first-line antibiotics rather than to a failure to change practice.
Use of drug benefit plan claims as the data source for a feedback program avoids some of the biases inherent in a voluntary reporting program such as a duplicate prescription system. In permits more rapid feedback than could be achieved through chart audits. It provides data on those who consent to participate as well as, in aggregate, those who decline. Promising the control group delayed feedback on the same drug class helped to control for the Hawthorne effect (alteration of one's routine behaviour in a setting of structured observation). [17] However, the Hawthorne effect is an intrinsic component of a feedback intervention, and controlling for it may have led to a more conservative estimate of the effect.
The absence of diagnostic data on the prescription claims made it impossible to evaluate the appropriateness of an individual prescription. The fact that we lacked sufficient information to judge participants' prescribing decisions may have contributed to the acceptability of the intervention, casting it as an informational program rather than as an audit. However, this nonjudgemental tone and simple program design came at the expense of a tightly defined endpoint - rate of appropriate prescribing - which could have been generated in a more rigorous audit.
The program that we describe included educational materials addressing common barriers to appropriate antibiotic prescribing and describing optimal prescribing behaviour. However, participants appeared to remain unclear on what was to be the target prescribing pattern. Despite explicit instructions to the contrary, many participants expressed reluctance to have their profile appear too different from that of the peer group, even if the difference were in the direction of more appropriate practice. This "herd effect," or the pursuit of safety in numbers, may reflect the difficulty physicians experience in dealing with the uncertainties inherent in clinical practice and their response to a medicolegal environment in which the usual standard of practice in the community is the measure against which they are judged. [18]
Our study has several limitations. We were not able to evaluate whether participants' changes in prescribing for their elderly patients with Ontario Drug Benefit program coverage were generalized to other patients in their practice. Since we do not have information on the numbers of patients presenting to participants' offices with symptoms suggesting an infection, we were not able to determine whether the rate of antibiotic prescribing was affected by the intervention. Our study also sheds no light on the effect of feedback on those who decline voluntary participation in such a program, especially since, as shown here, their practices may differ systematically from that of volunteers.
Highly effective interventions to modify prescribing behaviour have been characterized to include such features as local participation in guideline development, clearly defined behavioural goals, multifaceted programs that both enable and reinforce appropriate prescribing, and personal contact with the prescriber. [4] In contrast, the simple intervention that we describe was based on preexisting guidelines and won acceptance through a collegial tone and confidential feedback rather than direct personal contact. Its effectiveness in modifying prescribing behaviour appears to undermine the conventional wisdom that more elaborate programs are required to effect change. The relative magnitude of the change produced by such a mailed feedback and education program when compared to more intensive interventions is unknown. However, even if the effect in an individual case is modest, the fact that it can be readily applied on a broad basis may make its net benefit greater. Further study is needed to define the generalizability of these results to other drug groups, the durability of the observed benefit and the effect on prescribers who have not self-selected for the program. Nonetheless, this work offers encouragement to those who seek simple, low-cost, effective and acceptable interventions to modify physician behaviour.
We gratefully acknowledge Drs. C. David Naylor and G.M. Anderson, who assisted in the design of the program, and Dr. D.A. Redelmeier, who commented on an earlier version of the manuscript.
Competing interests: Dr. Hux is a Career Scientist of the Ontario Ministry of Health and receives salary support from the Institute for Clinical Evaluative Sciences in Ontario. The opinions, results and conclusions are those of the authors, and no endorsement by the Ministry of Health or the Institute for Clinical Evaluative Sciences is intended or should be inferred.
This article has been peer reviewed.
Reprint requests to: Dr. Janet E. Hux, G-106, Institute for Clinical Evaluative Sciences, 2075 Bayview Ave., Toronto ON M4N 3M5; fax 416 480-6048; [email protected]
References
In this issue
Article tools
Jump to section
Related Articles
Cited By...
- Ambulatory Antibiotic Stewardship through a Human Factors Engineering Approach: A Systematic Review
- Improvement in Herpes Zoster Vaccination in Patients with Rheumatoid Arthritis: A Quality Improvement Project
- Educational interventions to improve prescribing competency: a systematic review
- Determinants of Quinolone versus Trimethoprim-Sulfamethoxazole Use for Outpatient Urinary Tract Infection
- What is the role of quality circles in strategies to optimise antibiotic prescribing? A pragmatic cluster-randomised controlled trial in primary care
- Educating physicians to reduce benzodiazepine use by elderly patients: a randomized controlled trial
- Cluster randomised controlled trial of tailored interventions to improve the management of urinary tract infections in women and sore throat
More in this TOC Section
Similar Articles
