Canadian breast cancer guidelines: Have they made a difference?

Methods

This study was approved by the University of Manitoba Health Research Ethics Board. The population-based study cohort included all women living in Manitoba in whom in situ or invasive breast cancer was diagnosed from 1995 to 2003 inclusive. Patients were identified through the Manitoba Cancer Registry, which contains the records for more than 99.5% of all cases of cancer in Manitoba (Dr. Jeri Kostyra, formerly of CancerCare Manitoba: personal communication, 2005). Information on breast cancer staging, based on the American Joint Commission on Cancer (version 5),3 has been routinely collected for breast cancers diagnosed since Jan. 1, 1995. For women with multiple primary tumours, one was selected as the index tumour according to the following hierarchy: earliest diagnosis, highest stage and largest size.

The Manitoba Cancer Registry, which has been shown to contain highly accurate treatment data for cancer surgeries (unpublished data, 2006), was used to obtain demographic, tumour and treatment information for each case. The International Classification of Diseases (ninth revision, clinical modification [ICD-9-CM]) codes that were used to define the procedures are listed in Table 1. We identified and included procedures that occurred up to 3 months before or up to 1 year after diagnosis. If a woman had multiple procedures in the breast, the most extensive procedure within 1 year of the diagnosis was included.

There was no ICD-9-CM code for sentinel lymph node biopsy during the study period; therefore, data for this procedure are not recorded in the Manitoba Cancer Registry. During the study period, only a limited number of surgeons offered sentinel lymph node biopsy, and, until 2001, most biopsies were performed in concert with axillary node dissection. Since 2001, sentinel lymph node biopsy has been recorded on medical claim forms under tariff code 0438. An axillary assessment refers to patients who have had an axillary node dissection, sentinel lymph node biopsy or both procedures. An inappropriate axillary node dissection in ductal carcinoma in situ was defined as any axillary node dissection in the presence of breast-conserving surgery or where 6 or more nodes were identified following mastectomy.

Individual surgeons who were responsible for performing the procedures were identified through linkage of medical claims in the Manitoba Health data sets with Manitoba Cancer Registry data. Surgeon identity remained anonymous.

We examined 4 measures of care in this study: breast-conserving surgery in stage I–II breast cancer; axillary assessment (axillary node dissection, sentinel lymph node biopsy or both) for all invasive cancers, excluding stage IV; adequacy of axillary node dissection (< 10 nodes identified following axillary node dissection) for stage I–III; and axillary node dissection in ductal carcinoma in situ or stage 0. These measures and the related guideline recommendations (Appendix 1) are available at www.cmaj.ca/cgi/content/full/158/3/DC1. We selected these aspects of surgical care based on the existence of specific guidelines and the availability of reliable data for evaluation.

For each of the 4 measures, the procedure rates for individual surgeons who operated on 12 or more breast cancer cases per year were examined for clinically important variations across the study period. We accounted for confounding due to access to radiation facilities by examining breast-conserving surgery separately for Winnipeg and non-Winnipeg (e.g., rural) surgeons and patients.

The annual province-wide crude rates of breast cancer surgery and the coefficients of variation among surgeons in Manitoba were plotted for each measure from 1995 to 2003 to assess the presence of an inflection point following introduction of the guidelines in 1998.

In order to determine whether there was a statistical association between the introduction of the guidelines and the province-wide rates of surgical care over time, we used a generalized linear regression model with generalized estimating equations4 to test for significant differences in the province-wide rates between the pre- (1995–1997) and post- (1999–2003) guideline periods for each of the 4 measures (data from 1998 was excluded). This type of population-average model is appropriate to test for differences in the average rate of change over time.

We stratified the observations in the population-average models based on age, Regional Health Authority area of residence and pathology summary stage. The dependent variable was the number of events (breast-conserving surgery, axillary assessment, adequacy of axillary node dissection or axillary node dissection in ductal carcinoma in situ) in each population stratum.

We used a Poisson distribution to model axillary node dissection in ductal carcinoma in situ and a negative binomial distribution to model province-wide rates for the other 3 measures.

We used a generalized linear model with random surgeon effects to test for an association between guideline introduction and variations among surgeon rates over time. These subject-specific models were used to test for significant variations in surgery rates among surgeons and to test for significant differences in subject-specific surgical rates between the pre-and post-guideline periods for each measure.5 The Poisson distribution was selected to model surgeon rates for all measures because it provided the best fit to these data. Preliminary analyses of the data revealed little variation in surgeon-specific rates of change over time; therefore, a random-effects model with a random slope for year was not considered to be an appropriate choice to model these data.

We included the natural logarithm of the total population for each stratum as an offset variable in the population-average models, and the natural logarithm of the total population seen by each surgeon was included as an offset variable in the surgeon-specific models. When modelling breast-conserving surgery, axillary assessment, adequacy of axillary node dissection and axillary node dissection in ductal carcinoma in situ, we defined the total population as the total number of breast-conserving surgeries (without follow-up mastectomy) and mastectomies performed, the total number of patients with stage I–III breast cancer, the total number of axillary node dissections performed and the total number of patients with ductal carcinoma in situ respectively.

Both the population-average and subject-specific generalized linear regression models initially contained the fixed effects of year, pre-or post-guideline period, and year by pre-or post-guideline period. If the interaction model term was not statistically significant at α = 0.05, then a more parsimonious model that contained main effects was specified.

Results

There were 7528 breast cancer cases recorded in the Manitoba Cancer Registry from 1995 to 2003 inclusive. Selecting one tumour per woman left 7022 patients in the cohort. Patient, tumour and treatment characteristics are summarized in Table 2.

The breast cancer surgery rates for individual surgeons indicated that there was wide variation for each of the 4 measures of care across the study period. Descriptive statistics for each measure are provided in Table 3. Of note, axillary node dissection in ductal carcinoma in situ was a relatively low-volume procedure, with a median of 14 cases per surgeon over the study period; however, the number of cases per surgeon varied from 6 to 77. In conjunction with an inappropriate axillary node dissection in ductal carcinoma in situ, 70% and 30% of primary surgeries to the breast were mastectomies and breast-conserving surgeries respectively.

The subject-specific generalized linear regression models showed that there were statistically significant variations in care among surgeons for 3 of the 4 measures: breast-conserving surgery, axillary assessment and adequacy of axillary node dissection (p < 0.001). The variation of the random intercept was not significantly different from zero for axillary node dissection in ductal carcinoma in situ (p = 0.08).

The rates for each measure appeared to be unaffected by the introduction of the guidelines in 1998 except for the rate of axillary node dissection in ductal carcinoma in situ, which declined following release of the guidelines (Fig. 1). The results of the population-average generalized linear regression models revealed no significant differences in province-wide rates between the pre-and post-guideline period for breast-conserving surgery (p = 0.09), axillary assessment (p = 0.76), axillary node dissection in ductal carcinoma in situ (p = 0.09) and adequacy of axillary node dissection (p = 0.54).

• Download figure
• Open in new tab
• Download powerpoint

Fig. 1: Province-wide rates of breast cancer surgery, 1995 to 2003. AND = axillary node dissection, DCIS = ductal carcinoma in situ. Dashed vertical line indicates guideline introduction.

Consistent with what the province-wide rates showed, the subject-specific generalized linear regression models revealed that the rates by surgeon did not differ significantly between the pre-and post-guideline periods for axillary assessment (p = 0.12), axillary node dissection in ductal carcinoma in situ (p = 0.33) and adequacy of axillary node dissection (p = 0.78). However, the period main effect for rates of breast-conserving surgery by surgeon could not be interpreted as a simple difference between periods because the interaction of year by period was significant (p = 0.017). In particular, surgeon-specific rates of breast-conserving surgery increased in the years before introduction of the guidelines but remained relatively constant after their introduction. Thus, there was a statistically significant point of inflection in breast-conserving surgery rates following guideline introduction, but not in the expected (increasing) direction.

Fig. 2 plots the coefficients of variation among surgeons for each measure of care over the study period and is included as an illustration of inter-surgeon variations over time. Guideline introduction was not associated with any changes in the coefficients of variation for the measures studied. The number of cases of ductal carcinoma in situ was small; consequently, the coefficient of variation for the measure of axillary node dissection in ductal carcinoma in situ was not included.

• Download figure
• Open in new tab
• Download powerpoint

Fig. 2: Coefficients of variation among surgeons for 3 measures of surgical care from 1995 to 2003. Note: The number of cases of ductal carcinoma in situ was small; consequently, the coefficient of variation for the measure of axillary node dissection in ductal carcinoma in situ was not included. Dashed vertical line indicates guideline introduction.

To investigate whether differences in variation between the pre-and post-guideline periods existed, subject-specific generalized linear regression models were specified separately for the 2 periods, and the variation of the random intercepts for the 2 periods were compared for each measure. None of the examined measures demonstrated differences in variation between the periods. Specifically, the measures of breast-conserving surgery, axillary assessment and adequacy of axillary node dissection each demonstrated significant variation among surgeons in both the pre-and post-guideline models, and axillary node dissection in ductal carcinoma in situ did not show significant variation among surgeons in either model.

Interpretation

Despite evidence of clinically important variations in procedure rates among surgeons for all 4 of the measures of care studied and the statistically significant variations among surgeons for all but 1 of the measures (axillary node dissection in ductal carcinoma in situ), introduction of the Canadian breast cancer guidelines was not associated with any significant changes in province-wide procedure rates or in variations among surgeons in Manitoba. There may have been an improvement in the rate of axillary node dissection in ductal carcinoma in situ following introduction of the guidelines; however, this difference was not statistically significant, which may have been due to the small number of cases.

Previous studies using the same measures that we examined (breast-conserving surgery,6^–8 axillary assessment in invasive9 or noninvasive10 disease and the adequacy of axillary node dissection11) have also reported variations in breast cancer surgery rates that were considered clinically important and that are likely to be unacceptable to health care providers, administrators and, most importantly, patients. In contrast, other studies have reported post-guideline changes in surgical breast cancer care.12^,13 The analyses performed in these studies, however, were based on 2 discrete points in time or on a comparison of regions with and without guidelines.14 By examining trends over time within the same region, our study was able to more fully examine whether the guidelines were related to changes in clinical practice patterns.

There are limits to what can be ascertained through administrative data. The Manitoba Cancer Registry is a highly accurate source of information about breast cancer surgeries (unpublished data, 2006); however, errors in coding can result in incorrect or unrecorded procedures. In this analysis, we were careful to prevent incorrect interpretations by examining cohorts of patients with similar access to radiotherapy for breast-conserving surgery and by setting a 6-node minimum for identifying an inappropriate axillary node dissection following a mastectomy for ductal carcinoma in situ. Despite this, some observed variation that is attributed to surgeon preferences may actually be the result of patient or tumour factors that cannot be accounted for using administrative data.

We acknowledge that the statistical power of our study to detect minimal clinically important differences could be low. From a technical perspective, in the design stage of the study we considered power calculations for detecting minimal clinically important differences. However, given that nonlinear random-effects models have rarely been used in similar epidemiological investigations, reasonable estimates for the level-1 and level-2 variances and residual error, which are necessary to compute power for these models, were not available in the literature. From a more practical perspective, because we analyzed the entire population of breast cancer patients in Manitoba from 1995 to 2003, it is not possible to increase the number of observations available for analysis, as would be possible for survey or experimental data.

The Canadian breast cancer guidelines were rigorously developed and were disseminated through various media to promote uptake. Moreover, the guidelines have been regularly updated. Little has been done, however, to complete the life cycle of the guidelines in terms of implementation and evaluation of practice, as was recommended by Iscoe at their inception.15 Although there is some support in the literature for the ability of evidence-based guidelines to change patient outcomes and practices,16 these changes typically occur only when guidelines are introduced in the context of rigorous evaluations or implementation strategies.17 Development and dissemination alone are insufficient to change practices.18 Health Canada, as the main sponsor of the breast cancer guidelines, may want to re-evaluate its role in ensuring effective implementation and evaluation, so that the guidelines have the desired effect of reducing variation in the way that breast cancer is treated across Canada.

The guidelines provide important information to health care providers and patients, and they make a significant contribution toward quality assurance in breast cancer care. The present study shows that the majority of breast cancer patients in Manitoba received surgical care consistent with what is recommended in the guidelines. However, important variations in care exist among surgeons. With the possible exception of axillary node dissection in ductal carcinoma in situ, the guidelines have not had an observable effect on the rate of breast cancer surgery or on the degree of variation among surgeons for the surgical care of breast cancer in Manitoba. These guidelines provide the best measure of acceptable practice; thus it is of specific interest that they be thoroughly implemented and consistently evaluated to provide all breast cancer patients with the best care possible.

@ See related article page 785

Appendix 1