CMAJ's impact factor: room for recalculation

The creation of citation indexes brought symmetry to the study of scientific developments. Whereas the reference lists appended to published articles allow scientific advances to be tracked backward to their earlier conceptual origins, a citation index allows one to follow the career of scientific papers forward to establish their role in subsequent progress. The indexing of citations therefore permits sociologists and historians to see how "the cloth of science is woven."1

Citations are deemed the currency of science. Citation indexes are used to assess scientific impact,[2, 3] and algorithms based on citation data can corroborate and even forecast Nobel prize awards.4 Citation indexes are also used to determine the "impact factor" and related indicators of the performance of scientific journals within their particular fields.3 This editorial briefly details CMAJ's recent performance in relation to top-performing journals of general and internal medicine and anticipates changes in the near future. In another article in this issue5 (page 979) Eugene Garfield, Chairman emeritus of the Institute for Scientific Information (ISI) in Philadelphia, comments on the uses and abuses of citation data and indices.

A journal's impact factor expresses the relation between the volume of substantive scientific articles that it publishes and the frequency with which the journal is cited. For example, according to the ISI's Journal Citation Reports,6 CMAJ published a total of 350 substantive papers in 1995 and 1996. These represent the denominator of the ratio used to determine our impact factor. In 1997, 556 citations were made in the medical literature to items that had been published in CMAJ during 1995 and 1996. CMAJ's impact factor for 1997 was therefore 1.6. This compares with impact factors ranging from 5.0 to 27.8 for the 6 top-ranking journals in general and internal medicine (Table 1).

Numerators and denominators used to calculate impact factor vary widely from journal to journal, sometimes with interesting effect. For example, the impact-factor ranking of the Journal of Investigative Medicine ahead of the BMJ appears incongruous given the respective publication volumes of these journals. An alternative measure of impact, namely absolute citation counts3 (i.e., the number of citations in the current year to items in the journal published in any previous year) shows the BMJ to be well ahead not only of the Journal of Investigative Medicine but also of the Annals of Internal Medicine (Table 1). CMAJ received a total of 4129 citations in 1997. Admittedly, absolute citation counts tend to favour older journals and those that publish more papers.3

According to Journal Citation Reports the number of substantive scientific items (i.e., "source" items counted in the impact-factor denominator) published annually by the Lancet and by CMAJ almost doubled from 1996 to 1997 (Table 1). The change in the number of source items reported for the Lancet appears to be related mainly to the introduction of a Research Letters section in 1997. As these are short reports they are less likely to be cited, and thus we may, unfortunately, predict a decline in the Lancet's impact factor.

We were curious about the increase in the number of source items reported by Journal Citation Reports for CMAJ. Journal Citation Reports defines substantive articles as original research articles and review articles. Excluded are letters to the editor, editorials, book reviews, news items and the like. Hand-counting the number of substantive scientific papers published in 1997, we arrived at only 175, not the 303 reported by Journal Citation Reports (Table 1). Similarly, reviewing all issues of CMAJ in 1995 and 1996, we could identify only 176 articles that reported original research or were program descriptions or review articles. Using this number as our denominator, we arrived at an impact factor of 3.2. Because of the difficulty of classifying some items (such as a long letter on ethics, for example, or a supplement that included 9 different subsections) the denominator may have been as high as 208, giving an impact factor of 2.7. But CMAJ certainly did not publish 350 scientific papers and reviews in 1995 and 1996. According to our calculations, the number of substantive scientific articles published did rise markedly, from 97 in 1995 and 111 in 1996 to 175 in 1997. This reflects our efforts to increase the clinical relevance of the journal.7 However, there was an error in the tally reported by Journal Citation Reports.

Subsequent discussions with Janet Robertson, editor of Journal Citation Reports, revealed that the denominator used to calculate CMAJ's impact factor was inflated by the inclusion of reports in our News and Features section as research papers or clinical review articles (Janet Robertson, ISI, Philadelphia, Pa: personal communication, 1999). This problem will be corrected beginning in the current year, so that the CMAJ impact factor as published in 2001 will not need to be recalculated.

There are instances in various types of journals where the presentation of different article types makes accurate classification difficult for the ISI. The Institute attempts to count only the truly scientific papers and review articles and welcomes advice from editors on how to do this for their particular journal.

So what does an impact factor of 3.2, 2.7, or whatever the case may be, really mean? It means that we need to do better and to keep trying to get better papers. CMAJ is well behind other major journals internationally, and many Canadian authors continue to send papers to foreign journals for publication. By correcting our impact factor we hope to attract some of these authors to our pages and in so doing make the journal more informative, relevant and interesting for readers.