Volume of screening mammography and performance in the Quebec population-based Breast Cancer Screening Program

Methods

Between May 1998 and December 2000, 321 985 women had a first screening in the PQDCS. (In this report, we use “screening” to refer to “mammography screening examination”; each screening includes 4 radiographic views; 1 screening corresponds to 1 woman screened.) Of these women, 14 671 were excluded from the study: 13 966 because of breast symptoms (e.g., lump, nipple inversion, nipple discharge), 522 because of prior mastectomy and 183 because of absence of identifying information about the radiologist who read the mammograms. Among the remaining 307 314 asymptomatic women, the mammograms were read as abnormal for 33 938: 1709 (5.6 per 1000) who received a diagnosis of screening-detected in situ or invasive breast cancer within 1 year after the initial abnormal screening result9 and 32 229 (10.5%) who did not receive such a diagnosis and therefore had a false-positive initial reading.

All women included in this analysis signed a consent form agreeing to participate in the PQDCS, which includes transmission of their data to a central database for analysis. More than 90% of those screened sign the form.10 The Commission d'accès à l'information du Québec authorized the use of this database by the Ministère de la Santé et des Services sociaux du Québec (MSSS). In turn, the MSSS mandated analysis of the data by the PQDCS evaluation team.

Information on risk factors for breast cancer is routinely collected with a standardized questionnaire and captured in the province-wide PQDCS information system, which contains the following variables on participants: age, parity, age at first childbirth, menopausal status, age at menopause, history of breast cancer among first-degree relatives, use of hormone replacement therapy, height, weight, mammographic rating of breast density, previous mammography, history of breast reduction or implant, and previous breast aspiration or biopsy.

We took into account several characteristics of the radiologists: sex, year of certification in radiology, number of practices and type (hospital or private clinic), annual volume of screenings, abnormal-recall rate and abnormal-recall rate of colleagues. Sex and year of certification were derived from the Quebec medical directory for the year 1999/2000.11 Other variables were extracted from the PQDCS information system. For a given radiologist, type of practice and abnormal-recall rate of colleagues were those of the centre where he or she performed the most examinations each year.

We estimated a radiologist's screening volume from the total number of screenings interpreted by the radiologist and the time between the first and last readings, as registered in the PQDCS information system for that individual during the study period. The average monthly volume was then projected over 12 months. For example, a radiologist who interpreted 2000 screenings between Sept. 1, 1998, and Dec. 31, 2000 (28 months), had an estimated annual volume of 857. We estimated the annual screening volume of each facility similarly, using data in the PQDCS information system.

We analyzed the data for the 307 314 eligible women with a case–control approach. First, we analyzed breast cancer detection rates by comparing the 1709 women with screening-detected breast cancer and a 10% simple random sample of those without such cancer (n = 30 560), as no greater precision was offered by use of a larger sample. Second, we analyzed false-positive rates by comparing the 3159 women with false-positive readings and the 27 401 others in the random sample. In these comparisons, the odds ratios can be interpreted as approximations of the breast cancer detection rate ratios and the false-positive rate ratios.12^,13

We used logistic regression for the multivariate analyses. We evaluated the association of radiologists' screening volume with breast cancer detection and false-positive rates in models, including characteristics of women and radiologists as covariates. Trends in outcomes according to screening volume were tested using the median value within each category of this variable. We used the same strategy to test the association of facilities' screening volumes with breast cancer detection and false-positive rates. Finally, we assessed the independent contribution of the radiologist and the screening facility by fitting models that included both variables simultaneously, after ruling out colinearity. In accordance with the multilevel structure of the data,14 we adjusted the confidence intervals (CIs) for the intrareader and intraclinic correlation. We used a 5% level of statistical significance throughout, and all tests were 2-sided.

Results

The distribution of most breast cancer risk factors was similar among the 1709 women with screening-detected breast cancer (1297 invasive and 349 in situ cancers; unknown status in the other 63 cases) and the 30 560 women without this disease (Table 1). For instance, the mean age at screening was 59 and 58 years, the mean body mass index 27 and 26 kg/m² and the mean age at first childbirth 25 and 24 years, respectively. The proportions of women with a family history of breast cancer, with a previous breast aspiration or biopsy and with dense breasts were higher among the women with than among those without breast cancer.

In contrast, the 3159 women with false-positive mammograms were similar to the 27 401 women with normal mammograms with respect to most breast cancer risk factors (Table 1) except that, compared with the women who had normal readings, fewer of the women with false-positive readings reported previous mammography, more reported a previous breast aspiration or biopsy, and more had dense breasts.

The screening mammograms were interpreted by 275 radiologists working in 68 accredited facilities. Most of the radiologists (73%) were male, and 35% had received their radiology certification recently (between 1990 and 2000). About two-thirds (65%) were working primarily in private clinics. About half (53%) screened at a single facility; 6% screened at 4 or more facilities. The mean annual screening volume was 581 (range 16–2878) per radiologist and 2279 (range 307–7386) per facility.

Breast cancer detection rate ratios and 95% CIs, according to screening volume of either radiologists or facilities, are shown in Table 2. The radiologists' caseload did not seem to influence their ability to detect invasive or in situ breast cancer. The same was true for small invasive cancers (1 cm or less in diameter) (data not shown). By contrast, cancer detection was associated with the number of screenings performed in the facility. For all types of cancer combined, the adjusted detection rate ratio comparing the facilities performing at least 4000 screenings per year with those performing fewer than 2000 was 1.28 (95% CI 1.07– 1.52). The trend according to facility size was highly statistically significant (p = 0.004).

The false-positive rate ratio decreased significantly (p value for trend = 0.001) with increasing screening volume of radiologists (Table 3). The false-positive rate ratio comparing radiologists reading 1500 or more screenings per year with those reading fewer than 250 was 0.53 (95% CI 0.35–0.79). By contrast, the screening volume of facilities was not associated with false-positive readings: the false-positive rate ratio was 1.20 (95% CI 0.94–1.51) for those performing 4000 or more screenings per year compared with those performing fewer than 2000.

Table 4 shows the combined association of radiologists' and facilities' caseloads with performance. Radiologists who worked in facilities performing a greater number of screenings per year had higher detection rates than those who worked in facilities performing fewer, and this was true for all radiologists working in high-volume facilities, irrespective of their individual screening volume. In contrast, the false-positive rates decreased with increasing radiologist caseload, and this trend was clearer among those who worked in larger facilities.

Interpretation

Our results support the notion that radiologists' and facilities' screening volumes are independently associated with performance and that their effects may be complementary. In our analysis, radiologists who read larger numbers of screening mammograms and worked in facilities performing larger numbers of screenings tended to have higher breast cancer detection rates while maintaining lower false-positive rates than radiologists who performed fewer readings and worked in facilities performing fewer screenings.

A few studies15^,16^,17^,18^,19 have examined the association of radiologists' interpretation volume with their performance. Four of them15^,16^,17^,18 were conducted as experiments; since performance in a test environment appears to be unrelated to performance in actual practice,20^,21 the relevance of the results of these studies to our purposes is questionable. Only Kan and colleagues19 evaluated the relation of radiologists' reading volume with performance in an operating population-based breast cancer screening program, like ours. They showed a trend, although not significant, to improved cancer detection with greater reading volume. They also showed a U-shaped relation between abnormal-recall rate (a proxy for false-positive rate) and radiologists' screening-mammography volume. In that study, conducted in British Columbia, the smallest and largest categories of annual screening volume were < 2000 and 4000–5199; in our analysis the corresponding volumes were < 250 and 1500–2878. In addition, Kan and colleagues could adjust only for age and history of previous screening, and they did not study the relation of facility screening volume to performance.

Our data suggest an association between a centre's screening volume and its cancer detection rate but not its false-positive rate. Blanks and associates22 observed a significant increase in invasive breast cancer detection rates and a decrease in abnormal-recall rates with increasing facility screening volume. Yankaskas and coworkers23 also observed lower recall rates with increasing facility screening volume but saw no change in sensitivity. Neither group took into account radiologists' reading volume.

Our study had some limitations. First, there were few radiologists or facilities with large numbers of screenings, and this limited our ability to compare our results with those in settings with higher radiologist or facility screening volumes. Second, our study covered only the first 2 years of the PQDCS, which might not be representative of current functioning. Third, only screening-mammography volumes of radiologists and facilities were available to us, whereas accreditation criteria are based on both screening and diagnostic experience. Finally, other important determinants of performance, such as double reading, participation in teaching or research, daily quality-control procedures within facilities and specific training of the radiologists in the interpretation of screening mammograms, could not be taken into account.

In conclusion, radiologists' and facilities' screening volumes appear to have independent and complementary influences on performance as measured by rates of breast cancer detection and false-positive readings. Although the average crude breast cancer detection rates met the minimum Quebec standard of 5 cancers per 10007 in all categories of volume shown for radiologists and facilities, the overall performance of screening mammography seems to be maximized when screenings are performed in larger centres and when, in these centres, mammograms are read by radiologists who interpret a large volume of films.

𝛃 See related article page 210