Evidence to action: a tailored multifaceted approach to changing family physician practice patterns and improving preventive care

The Canadian Task Force on Preventive Health Care (formerly the Canadian Task Force on the Periodic Health Examination) has established guidelines for the delivery of preventive care1^,2 that are supported by clinical evidence as effective. Evidence-based guidelines are not self-implementing.3^,4^,5 Changing physicians' long-held patterns of behaviour and the environments in which they work is complex and difficult. Unless the barriers to change can be overcome and action taken to improve compliance, efforts to develop evidence-based guidelines will be wasted.6

Programs that address physician knowledge alone, such as traditional continuing medical education and dissemination of guidelines, are insufficient to change practice behaviour.3^,4^,7^,8 In addition, single interventions are less likely to result in significant improvement of practice behaviour than interventions using 2 or more strategies in an intensive combined intervention.4^,9^,10^,11^,12^,13 A systematic review of 102 randomized controlled trials showed that single interventions such as educational materials, reminder systems, and audit and feedback had modest or negligible practical effects when used alone.10 However, combined intervention strategies resulted in significant changes in physician behaviour and health outcomes.4^,10 Bero and colleagues5 reviewed 18 systematic reviews on improvement to professional performance and determined that multifaceted interventions were effective. An organized system of appropriate sets of tools can increase preventive care in primary care practice.14^,15^,16^,17^,18^,19^,20^,21^,22^,23^,24^,25

By its nature, facilitation is multifaceted. In the United Kingdom specially trained nurse facilitators organized preventive care in busy practitioners' offices using approaches such as academic detailing, chart audit and feedback for the prevention and early detection of cardiovascular disease.26^,27 In the United States the health facilitator model was shown to be efficacious in establishing office routines and significantly improved the provision of services for the early detection and prevention of cancer.15 Hulscher and associates19 found that adapting the facilitator intervention by tailoring activities to the needs of the practice was effective in improving adherence for 6 of 10 office organizational guidelines.

Most studies on prevention have been done in a fee-for-service setting and have involved 1 or a few recommended preventive manoeuvres. None has measured the performance of inappropriate preventive measures or the rate of uptake of various methods to improve prevention over time. We describe a randomized controlled field trial of a tailored multifaceted intervention to improve preventive care in capitation-based family practices.

Methods

The intervention involved health service organizations (HSOs) in Ontario. HSOs are community primary care practices that have a payment system based primarily on capitation. There are 72 physician-sponsored HSOs at 100 different sites in Ontario, most of which are urban and located in the Toronto/Hamilton area.

Recruitment of HSO practices into the study involved repeated mailings followed, when necessary, by a telephone call from a physician recruiter. Recruitment continued until the required number of HSOs provided informed consent to participate in the study. Four HSOs located in remote areas were excluded because of cost, and the HSO in which we work was also excluded.

There were 20 practices per study group, for a total of 40 practices. This allowed the detection of a mean difference of 0.09 in the preventive performance index between the intervention and control groups with an α value of 0.05 and 80% power. The sample size for patient medical records was based on a precision of ± 0.02 with 95% confidence for the preventive performance index. A total of 4000 patient charts, or 100 patient charts per practice, were reviewed at both baseline and follow-up.

This was a clinical trial using facilitators to improve preventive care. The randomization process and progress through the study are shown in Fig. 1. After all baseline measurements were completed we randomly assigned practices to either the intervention or the control group. The primary care practice (1 to 6 doctors) was the unit of randomization and the unit of analysis. The primary study comparison was between the intervention and control practices with respect to changes from baseline (May 1997) to the end of the intervention period (Dec. 31, 1998) in the performance of 8 recommended and 5 inappropriate preventive manoeuvres. The study was approved by the Ottawa Civic Hospital Ethics Review Board.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 1: Randomization process and progress through the study. R = randomization.

The intervention involved 3 prevention facilitators who had master's degrees in community nursing and experience in facilitation. The facilitators completed a 30-week intensive training program before being assigned to intervention practices. The training involved course work, assignments and practical experience in an HSO practice setting.

Each facilitator had primary responsibility for up to 8 primary care practices. Practice assignment was done according to geographic proximity to the facilitators' residences. The facilitators had no interaction with the control practices. The latter were told that they were involved in a study on prevention but were not told which manoeuvres were being measured.

The intervention is described in detail elsewhere.28 The facilitators called each intervention practice, arranged an appointment with the lead physician and were introduced to the practice staff. They used 7 intervention strategies, identified from reviews of the literature, that were designed to change practice patterns and improve preventive care performance10^,11 (Box 1). They discussed the strategies with the physicians and practice staff, working with them to adapt the strategies to the practice needs and wishes. They provided references to physicians when appropriate and gave performance feedback using miniaudits. They provided management support to practices and followed a quality-improvement framework similar to that proposed by Leininger and collaborators18 (Box 2). During the study they organized a conference on disease prevention.

• Download figure
• Open in new tab
• Download powerpoint

Box 2.

• Download figure
• Open in new tab
• Download powerpoint

Box 1.

The intervention lasted 18 months. During this period each practice was visited an average of 33 (range 21–50) times, with an average visit length of 1 hour and 45 minutes.

The intervention used the recommendations of the Canadian Task Force on Preventive Health Care.2 Eight grade A and B, and 5 grade D preventive manoeuvres were chosen by a panel of practising family physicians to represent a broad spectrum for both male and female adult patients (Appendix 1). The nurse facilitators and the physicians in the intervention group were aware of the preventive manoeuvres under study but were instructed not to limit preventive care efforts to these manoeuvres.

We calculated 3 outcome measures: an overall index of preventive performance, an up-to-datedness index and an inappropriateness index. The preventive performance index was the proportion of eligible patients who received the recommended preventive manoeuvres less the proportion of eligible patients who received the inappropriate preventive manoeuvres. The up-to-datedness index was the proportion of recommended manoeuvres done, and the inappropriateness index was the proportion of inappropriate manoeuvres done.

We conducted a secondary analysis of each manoeuvre to show which were most affected by the intervention.

We collected outcome data from the time the study began to track the rate of improvement in prevention over time. We analyzed preventive performance after 9 months, 15 months and 18 months according to the date of the patient's last visit, as determined by the chart audit.

The facilitators conducted a prerandomization review of 100 patient charts per practice. Chart auditors reviewed 100 charts per practice at follow-up. Both recorded preventive manoeuvres performed during the previous 2 years. Preventive manoeuvres were excluded if there was any indication that they were done for diagnostic reasons. Both baseline and follow-up samples were chosen randomly, and the data were entered directly into a laptop computer. The chart auditors were blinded as to the status of the practices and the assessment of outcomes. Interrater reliability was determined through assessment of a sample of each audit by an independent reviewer. If more than 5 of 20 charts were coded incorrectly, the entire 100 charts were audited again. The concordance between auditors was 85.4% (kappa = 0.71) at baseline and 84.4% (kappa = 0.69) at follow-up.

The documentation of such preventive manoeuvres as counselling depended on physician charting, whereas other manoeuvres, such as mammography, were documented with a laboratory report in the chart. To ensure that changes in prevention performance in the intervention group were due to improved performance rather than improved charting, we administered a pre- and postintervention patient telephone survey for folic acid supplementation, smoking cessation counselling and hypertension treatment. We then compared the survey results with those of the chart audit.

The telephone interview was conducted with a random sample of 25 patients per 100 audited charts. Patients with a diagnosis of hypertension were asked whether they had been prescribed any medication, patients who smoked were asked whether they had been counselled to quit smoking, and female patients aged 19–36 were asked whether they had been counselled to take folic acid. All patients provided informed consent before completing the interview.

Analytical methods for the various end points in the study took into account the measurement level, frequency and nature of each of the outcome variables. We compared the intervention and control groups to determine whether there were any significant differences. Cross-tabulations using a 𝛘² test and Fisher's exact test were used to examine categorical data and compare groups. We used Student's t-test for independent groups for comparisons of continuous data. To test for significant differences in end points between the intervention and control groups, we analyzed end points using general linear model repeated-measures analysis of variance, where end points measured at baseline and follow-up were treated as within-subject factors or dependent variables, and the intervention group was the between-subjects factor or independent variable. Mean proportions and 95% confidence intervals (CIs) were computed. Significant interaction effects were further analyzed with a least-significant-difference post-hoc test to evaluate mean differences. We used general linear model analysis of variance to test for differences between the study groups in preventive performance index after 9 months, 15 months and 18 months of intervention. Finally, we used Student's t-test for independent groups to test for significant differences between the 2 groups in the mean proportion of patients who received 13 preventive manoeuvres according to chart audit and 3 preventive manoeuvres according to patient self-report.

Results

Of the 95 HSOs contacted, 49 were not randomized: 30 refused to participate, and 19 initially agreed to participate but did not sign consent. Thus, 46 practices (48.4%) were recruited, of which 23 were assigned to receive the intervention. One HSO, in the intervention group, was lost to follow-up because the practice had moved. Participating physicians were younger than nonparticipants (p < 0.001) and were more likely to be female (p = 0.001). However, the intervention and control practices did not differ significantly in any of the measured demographic characteristics (Table 1).

The mean proportions of eligible patients who received the preventive manoeuvres, as determined by chart audit, are shown in Table 2. For grade A and B manoeuvres, the improvement in performance was significantly higher for the intervention group than for the control group for folic acid supplementation (p = 0.014) and influenza vaccination (p = 0.012). For hypertension treatment, there was a significant improvement in the control practices (p = 0.009). For grade D manoeuvres, there was significant improvement in the intervention practices for blood glucose screening (p = 0.016) and proteinuria screening (p = 0.01). For prostate-specific antigen screening and routine chest radiography, there was a trend toward better improvement in the control practices.

Before the intervention the preventive performance index was similar for the intervention and control groups (31.9% [95% CI 27.3%–36.5%] and 32.1% [95% CI 27.2%–37.0%] respectively). At follow-up the corresponding values were 43.2% (95% CI 38.4%–48.0%) and 31.9% (95% CI 26.8%–37.0%). The absolute improvement in the intervention group was 11.5% (p < 0.001).

In the intervention group the up-to-datedness index was 52.3% (95% CI 48.6%–56.0%) before the intervention and 62.3% (95% CI 58.2%–66.4%) after the intervention. The corresponding values for the control group were 54.6% (95% CI 51.0%–58.2%) and 57.4% (95% CI 54.1%–60.7%). The absolute improvement in the intervention group was 7.2% (p = 0.008).

The intervention practices performed fewer inappropriate manoeuvres at follow-up than the control practices (inappropriateness index 19.1% [95% CI 15.6%–22.6%] v. 25.5% [95% CI 20.0%–31.0%]). The absolute improvement in the intervention group was 4.4% (p = 0.019).

Fig. 2 shows the improvement in preventive performance index over time for the intervention and control groups. Only in the last 3 months of the study were all the preventive manoeuvres adopted by the intervention practices. The interaction effect between the control and intervention groups over time was significant, with the last 3 months showing the most improvement (p < 0.001).

• Download figure
• Open in new tab
• Download powerpoint

Fig. 2: Mean index of preventive performance for 4497 eligible patients in the intervention and control practices over the intervention period.

The total number of patients contacted by telephone was 1150, or 100% of the target number. Overall, 1981 telephone calls were made, of which 779 (39.3%) were unsuccessful in reaching the patient; of the 1202 contacted, 52 (4.3%) refused to be interviewed. There was a higher absolute rate of folic acid supplementation and smoking cessation counselling indicated from the patient survey than from the chart audit, but a higher rate of hypertension treatment indicated from the chart audit than from the patient survey. The difference in overall prevention rates between the chart audit and the patient survey was the same before and after the intervention (Table 3).

Interpretation

Our rate of improvement in preventive care performance, 11.5%, is comparable to the rates reported for other multifaceted interventions to change preventive performance in which more than 5 preventive manoeuvres were targeted for change (9.2%,15 13.3%23 and 12.9%25). These results might appear modest; however, the potential effect of an improvement of 11.5% for 13 different preventive interventions on an entire population served by family physicians is considerable. In addition, the improvement in overall prevention may be an underestimate, since it was only in the last 3 months of the study that all the prevention tools were introduced. Comparison of data from the patient survey and the chart audit showed that it was unlikely that the changes were due to improvement in charting alone.

Our study has several strengths: the rigorous randomized controlled trial design, a primary outcome measure that considered a large number of both recommended and inappropriate preventive manoeuvres supported by the evidence-based guidelines of the Canadian Task Force on Preventive Health Care, data collection by both chart audit and patient survey, and the community practice setting. Audits were performed by the nurse facilitators before randomization, and follow-up audits were performed by different auditors, who were blinded as to the status of the practices and the assessment of outcomes. Both the facilitators and the chart auditors had high interrater reliability coefficients. Finally, to our knowledge, there are no previous reports of an intervention aimed at decreasing the use of inappropriate preventive manoeuvres. The ability to decrease the performance of such manoeuvres is novel and is important in that it may prevent harm to patients from unnecessary interventions.

There are several weaknesses to our study. The study practices were self-selected and not representative of all HSOs in Ontario. Physicians in nonparticipating HSOs were more likely to be older and male than physicians in the participating HSOs. The physicians in the intervention group knew what the outcome measures were, whereas those in the control group did not. This may have resulted in a slight cointervention effect. However, such an effect is unlikely since prevention is very difficult to improve using complex interventions such as ours, and the mere knowledge of outcome measures is unlikely to be a significant factor, especially over an extended period. For a comprehensive manoeuvre-by-manoeuvre comparison between the intervention and control groups, a larger sample would have been necessary. The small number of HSO practices in Ontario did not allow for this. Our inability to decrease prostate-specific antigen testing may have been due to contradicting guidelines and media pressure to perform this test. Finally, we cannot generalize our findings to other practices. However, it has been shown that in Ontario, fee-for-service and capitation-based primary care practices do not differ in their characteristics,30 preventive care performance31 or rates of hospital use.32

Conclusion

Changing medical practice to improve preventive care is complex. A tailored multifaceted approach delivered by nurse facilitators can significantly improve the preventive care performance of capitated primary care physicians. More research is needed to explain which elements of complex interventions work, the theory behind why they work, and whether they are cost-effective. The development and adoption of standards for methods and measurement tools in studies of prevention would help in comparing various intervention strategies. Finally, although governments and medical colleges focus mainly on continuing medical education to change physician behaviour and ensure quality of care, information transfer is not enough. The use of facilitators is more likely to change physician behaviour than traditional update courses.

Appendix 1