 Advertisement | ||||||||
 | ||||||||
|
||||||||
 |
 |
 |
 |
 |
 |
 |
||
|
Dr. Garfield is Chairman Emeritus, Institute for Scientific Information, Philadelphia.
I first mentioned the idea of an impact factor in 1955.1 At that time it did not occur to me that it would one day become the subject of widespread controversy. Like nuclear energy, the impact factor has become a mixed blessing. I expected that it would be used constructively while recognizing that in the wrong hands it might be abused.
In the early 1960s Irving H. Sher and I created the journal impact factor to help select journals for the Science Citation Index (SCI). We knew that a core group of highly cited large journals needed to be covered in the SCI. However, we also recognized that small but important review journals would not be selected if we depended solely on simple publication or citation counts.2 We needed a simple method for comparing journals regardless of their size, and so we created the journal impact factor.
The use of the term "impact factor" has gradually evolved, especially in Europe, to include both journal and author impact. This ambiguity often causes problems. It is one thing to use impact factors to compare journals and quite another to use them to compare authors. Journal impact factors generally involve relatively large populations of articles and citations. Individual authors, on average, produce much smaller numbers of articles.
A journal's impact factor is based on 2 elements: the numerator, which is the number of citations in the current year to any items published in a journal in the previous 2 years, and the denominator, which is the number of substantive articles (source items) published in the same 2 years. The impact factor could just as easily be based on the previous year's articles alone, which would give an even greater weight to rapidly changing fields. A less current impact factor could take into account longer periods. Alternatively, one could go beyond 2 years for the source items in the denominator, but then the measure would be less current.
All citation studies should be normalized to take into account variables such as field, or discipline, and citation practices. Citation density and half-life are also important variables. The citation density (mean number of references cited per article) would be significantly lower for a mathematics article than for a life sciences article. The half-life (number of years, going back from the current year, that cover 50% of the citations in the current year to the journal) of a physiology journal would be longer than that of a journal of molecular biology or astronomy. The impact factors currently reported by the Institute for Scientific Information in Journal Citation Reports (JCR) may not provide a complete enough picture for slower changing fields with longer half-lives. Nevertheless, when journals are studied within disciplinary categories, the rankings based on 1-, 7- or 15-year impact factors do not differ significantly, as was recently reported in The Scientist.3,4 In the first report the top 100 journals with the highest impact factors were compared;3 in the second report the next 100 journals were compared.4 When journals were studied across fields, the ranking for physiology journals as a group improved significantly as the number of years increased, but the rankings within the group did not. Hansen and Henrikson5 reported "good agreement between the journal impact factor and the overall [cumulative] citation frequency of papers on clinical physiology and nuclear medicine."
The impact factor calculations used by JCR tacitly imply that all editorial items in Science, Nature, the Journal of the American Medical Association, CMAJ, the British Medical Journal and The Lancet can be neatly categorized. Such journals publish large numbers of items that are neither traditional research nor review articles. These items (e.g., letters, news stories and editorials) are not included in JCR's calculation of impact, yet we all know that they may be cited. Indeed, the JCR numerator includes citations to any item published in these journals. The assignment of codes by article type is based on human judgement. A news story might be perceived as a substantive article, and a research letter might not be. Furthermore, no effort is made to differentiate clinical versus laboratory studies or, for that matter, practice-based versus research material.
There is a widespread but mistaken belief that the size of the scientific community that a journal serves affects the journal's impact. This assumption overlooks the fact that the larger the author and article pool for citing, the larger the number of published articles to share those citations. Many articles in large fields are not well cited, whereas those in small fields may have unusual impact. Therefore, the key determinants in impact are not the number of authors or articles in the field but, rather, the mean number of citations per article (density) and the half-life or immediacy of citations to a given journal. This distinction was discussed many years ago in an essay on "Garfield's constant".6 The size of a field, however, will determine the number of "super-cited" papers. While a few famous methodology papers achieve a high threshold of citation, thousands of other methodology papers do not achieve this distinction.
The time required to review manuscripts may also affect impact. If reviewing and publication are delayed, references to articles that are no longer current may not be included in the impact calculation. Even the appearance of articles on the same subject in the same issue of a journal may have an effect. Opthof recently showed how journal impact performance varies from issue to issue.
For greater precision, it is preferable to conduct item-by-item journal audits so that any differences in impact for these different types of editorial items can be taken into account.8 For a small number of journals a bias may be introduced by including in the numerator these extra citations to items that are not part of the denominator of source articles. Clearly, if the denominator is smaller than the actual number of published items, it will increase the journal's impact factor. This in turn may alter the rankings. However, most journals publish primarily substantive research or review articles. Therefore, statistical discrepancies are significant only in rare cases. The JCR data have come under some criticism for this reason among others.9
Most discrepancies are eliminated altogether in another database called the ISI Journal Performance Indicators (JPI) (www.isinet.com/products/rsg/jperfind.html). This annual compilation now covers citations from 1981 to 1998. Because the database links each source item to its citation in JPI, the impact calculations are more precise, in that citations are counted only for substantive items and it is possible to obtain impact measures covering longer periods. For example, the cumulated impact for CMAJ articles published in 1981 is 9.04 (derived by dividing the number of articles published in CMAJ that year [224] into the number of citations between 1981 and 1998 [2024]). Using similar data, I was able to calculate 7- and 15-year impact factors for the 200 high-impact scientific and medical journals mentioned earlier.3,4
In addition to helping libraries decide which journals to purchase, journal impact factors are also used by authors to decide where to submit their articles. As a general rule, the journals with high impact factors are among the most prestigious today. The perception of prestige is a murky subject. Some would equate prestige with high impact. However, some librarians argue that the numerator in the impact-factor calculation is in itself even more relevant. Bensman10 stated that this 2-year citation count is a better guide to journal significance and cost-effectiveness than is the impact factor.
Journal impact can also be useful in comparing expected and actual citation frequency. Thus, when ISI prepares a personal citation report it provides data on the expected citation impact not only for a particular journal but also for a particular year, because impacts change from year to year. For historical comparisons, a 1955 article cited 250 times might be considered a "citation classic," whereas the threshold for a 1975 article might be 400 and a 1995 article 1000.
The use of journal impact factors instead of actual article citation counts is probably the most controversial issue. Granting and other policy agencies often wish to bypass the work involved in obtaining actual citation counts for individual articles and authors. Recently published articles may not have had enough time to be cited, so it is tempting to use the impact factor as a surrogate, virtual count. Presumably the journal's impact and the mere acceptance of the paper for publication is an implied indicator of prestige and subsequent citation. Typically, when the author's bibliography is examined, a journal's impact factor is substituted for the actual citation count. Thus, use of the impact factor to weight the influence of a paper amounts to a prediction, albeit coloured by probabilities.
The assumption that any recent article cannot be evaluated may be wrong. Indeed, papers that achieve rapid impact are cited within months and certainly within a few years. This pattern of immediacy has enabled the ISI to identify "hot papers" in its bimonthly publication Science Watch. However, full confirmation of high impact is generally obtained 2 years later. The Scientist waits up to 2 years to select "hot papers" for commentary by authors. Most of these papers will eventually qualify as "citation classics."
Of the many conflicting opinions about impact factors, I believe that Hoeffel11 expressed the situation succinctly.
Impact Factor is not a perfect tool to measure the quality of articles but there is nothing better and it has the advantage of already being in existence and is, therefore, a good technique for scientific evaluation. Experience has shown that in each specialty the best journals are those in which it is most difficult to have an article accepted, and these are the journals that have a high impact factor. These journals existed long before the impact factor was devised. The use of impact factor as a measure of quality is widespread because it fits well with the opinion we have in each field of the best journals in our specialty.
|
SYMBOL See related article page 977
Correspondence to: Dr. Eugene Garfield, President and Editor-in-Chief, The Scientist, 3600 Market St., Philadelphia PA 19104, USA; fax 215 387-1266; garfield{at}codex.cis.upenn.edu
References
This article has been cited by other articles:
|
|
 |
|
  R. A. Brumback The Silver Jubilee: Journal of Child Neurology Turns 25 J Child Neurol, January 1, 2010; 25(1): 4 - 31. [PDF]
|
|
|
|
 |
|
 R. K. Goodyear, D. J. Brewer, K. S. Gallagher, T. J. G. Tracey, C. D. Claiborn, J. W. Lichtenberg, and B. E. Wampold The Intellectual Foundations of Education: Core Journals and Their Impacts on Scholarship and Practice Educational Researcher, December 1, 2009; 38(9): 700 - 706. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 G. Paraje, R. Sadana, and R. Salmela Collaboration and "Visibility" of Health Research in the Western Pacific Region Asia Pac J Public Health, April 1, 2009; 21(2): 128 - 136. [Abstract] [PDF]
|
|
|
|
 |
|
 T Jefferson, C Di Pietrantonj, M G Debalini, A Rivetti, and V Demicheli Relation of study quality, concordance, take home message, funding, and impact in studies of influenza vaccines: systematic review BMJ, February 12, 2009; 338(feb12_2): b354 - b354. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. Ramsdell, J. Lerman, D. Pickhardt, D. Feldman, J. Foster, and T. T. Houle Subspecialty Impact Factors: The Contribution of Pediatric Anesthesia and Pain Articles Anesth. Analg., January 1, 2009; 108(1): 105 - 110. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. L. Pool, M. Macy, S. Bells McManus, and J. Noh An Exploratory Investigation of Frequently Cited Articles From the Early Childhood Intervention Literature, 1994 to 2005 Topics in Early Childhood Special Education, November 1, 2008; 28(3): 181 - 189. [Abstract] [PDF]
|
|
|
|
 |
|
 G. Holden, K. Barker, L. Covert-Vail, G. Rosenberg, and S. A. Cohen Does Social Work Abstracts Work? Research on Social Work Practice, September 1, 2008; 18(5): 487 - 499. [Abstract] [PDF]
|
|
|
|
|
|
R. A. Brumback Worshiping False Idols: The Impact Factor Dilemma J Child Neurol, April 1, 2008; 23(4): 365 - 367. [PDF]
|
|
|
|
 |
|
 M. Rossner, H. Van Epps, and E. Hill Show Me the Data J. Gen. Physiol., December 31, 2007; 131(1): 3 - 4. [Full Text] [PDF]
|
|
|
|
 |
|
 M. Rossner, H. Van Epps, and E. Hill Show me the data J. Exp. Med., December 24, 2007; 204(13): 3052 - 3053. [Full Text] [PDF]
|
|
|
|
 |
|
 M. Rossner, H. Van Epps, and E. Hill Show me the data J. Cell Biol., December 20, 2007; 179(6): 1091 - 1092. [Full Text] [PDF]
|
|
|
|
 |
|
 T. A. Winnick Trends in attention to complementary and alternative medicine in the American medical literature Health (London) , July 1, 2007; 11(3): 371 - 399. [Abstract] [PDF]
|
|
|
|
|
|
G. Williams Should we ditch impact factors? BMJ, March 17, 2007; 334(7593): 568 - 568. [Full Text] [PDF]
|
|
|
|
|
|
G. Holden, G. Rosenberg, K. Barker, and P. Onghena An Assessment of the Predictive Validity of Impact Factor Scores: Implications for Academic Employment Decisions in Social Work Research on Social Work Practice, November 1, 2006; 16(6): 613 - 624. [Abstract] [PDF]
|
|
|
|
 |
|
 M. A. Rodriguez, J. Bollen, and H. Van de Sompel The convergence of digital libraries and the peer-review process Journal of Information Science, April 1, 2006; 32(2): 149 - 159. [Abstract] [PDF]
|
|
|
|
 |
|
 J. Cheek, B. Garnham, and J. Quan What's in a Number? Issues in Providing Evidence of Impact and Quality of Research(ers) Qual Health Res, March 1, 2006; 16(3): 423 - 435. [Abstract] [PDF]
|
|
|
|
 |
|
 S. Hakkalamani, A. Rawal, M. S. Hennessy, and R. W. Parkinson The impact factor of seven orthopaedic journals: FACTORS INFLUENCING IT J Bone Joint Surg Br, February 1, 2006; 88-B(2): 159 - 162. [Full Text] [PDF]
|
|
|
|
 |
|
 E. V Bernstam, J. R Herskovic, Y. Aphinyanaphongs, C. F Aliferis, M. G Sriram, and W. R Hersh Using Citation Data to Improve Retrieval from MEDLINE JAMIA, January 1, 2006; 13(1): 96 - 105. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 B. Vincent, M. Vincent, and C. G. Ferreira Making PubMed Searching Simple: Learning to Retrieve Medical Literature Through Interactive Problem Solving. Oncologist, January 1, 2006; 11(3): 243 - 251. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 W. T. Obremskey, N. Pappas, E. Attallah-Wasif, P. Tornetta III, and M. Bhandari Level of Evidence in Orthopaedic Journals J. Bone Joint Surg. Am., December 1, 2005; 87(12): 2632 - 2638. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. D. Lee, A. Cassano-Pinche, and K. J. Vicente Bibliometric Analysis of Human Factors (1970-2000): A Quantitative Description of Scientific Impact Human Factors: The Journal of the Human Factors and Ergonomics Society, January 1, 2005; 47(4): 753 - 766. [Abstract] [PDF]
|
|
|
|
|
|
J. Lauritsen and A. M. Moller Publications in Anesthesia Journals: Quality and Clinical Relevance Anesth. Analg., November 1, 2004; 99(5): 1486 - 1491. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 N. S. Murali, H. R. Murali, P. Auethavekiat, P. J. Erwin, J. N. Mandrekar, N. J. Manek, and A. K. Ghosh Impact of FUTON and NAA Bias on Visibility of Research Mayo Clin. Proc., August 1, 2004; 79(8): 1001 - 1006. [Abstract] [PDF]
|
|
|
|
 |
|
 R. SoRelle Cardiovascular News Circulation, June 29, 2004; 109(25): e9054 - e9055. [Full Text]
|
|
|
|
 |
|
 N. Kovacic and A. Misak Author self-citation in medical literature Can. Med. Assoc. J., June 22, 2004; 170(13): 1929 - 1930. [Full Text] [PDF]
|
|
|
|
|
|
J. Maskalyk Medical journals and global medicine Can. Med. Assoc. J., January 6, 2004; 170(1): 65 - 65. [Full Text] [PDF]
|
|
|
|
|
|
A. P. Kurmis Understanding the Limitations of the Journal Impact Factor J. Bone Joint Surg. Am., December 1, 2003; 85(12): 2449 - 2454. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
R. A. Brumback Impact of the Journal of Child Neurology: 2002 Data J Child Neurol, November 1, 2003; 18(11): 795 - 797. [Abstract] [PDF]
|
|
|
|
 |
|
 L. M. Hasbrouck, J. Taliano, J. M. Hirshon, and A. L. Dannenberg Use of Epidemiology in Clinical Medical Publications, 1983-1999: A Citation Analysis Am. J. Epidemiol., March 1, 2003; 157(5): 399 - 408. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. F. Rogers Impact Factor: The Numbers Game Am. J. Roentgenol., March 1, 2002; 178(3): 541 - 542. [Full Text] [PDF]
|
|
|
|
 |
|
 V. Hachinski The Impact of Impact Factors Stroke, December 1, 2001; 32(12): 2729 - 2729. [Full Text] [PDF]
|
|
|
|
 |
|
 P J Van Diest, H Holzel, D Burnett, and J Crocker Impactitis: new cures for an old disease J. Clin. Pathol., November 1, 2001; 54(11): 817 - 819. [Full Text] [PDF]
|
|
|
|
|
|
J. Hoey Improved ranking for CMAJ Can. Med. Assoc. J., March 1, 2001; 164(6): 747 - 747. [Full Text] [PDF]
|
|
|
|
|
|
P. A. Singer Medical journals are dead. Long live medical journals Can. Med. Assoc. J., February 1, 2000; 162(4): 517 - 518. [Full Text] [PDF]
|
|
|
|
|
|
S. van Zanten Impact of abstracts and short reports Can. Med. Assoc. J., February 1, 2000; 162(4): 489 - 489. [Full Text] [PDF]
|
|
|
|
|
|
K. S. Joseph and J. Hoey CMAJ's impact factor: room for recalculation Can. Med. Assoc. J., October 1, 1999; 161(8): 977 - 978. [Full Text] [PDF]
|
|
Read all eLetters
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | CURRENT ISSUE | PAST ISSUES | COLLECTIONS | HELP | SEARCH |
|
|||||
|
Copyright 1995-1999, Canadian Medical Association. All rights reserved. ISSN 1488-2329
(e) 0820-3946 (p)
All editorial matter in CMAJ represents the opinions of the authors and not necessarily those of the Canadian Medical Association. |
|||||
