There are thousands of medical journals published worldwide, and it seems busy doctors and academics sometimes have trouble determining which ones are worth reading, which ones are worth skimming and which ones shouldn’t even have the word “journal” in their titles. Of course, there are professionals with a skillset particularly suited to helping users of medical journals separate the roses from the ragweed: research librarians.
After news broke in April that academic publishing giant Elsevier published an industry-funded fake journal in Australia (The Australasian Journal of Bone and Joint Medicine), research librarians flocked to the Internet to discuss the topic of publications designed to trick “average readers” into believing they are legitimate, peer-reviewed medical journals. “This makes me think, a librarian, especially a medical librarian, should not be an ‘average reader’, not even if ‘average’ means ‘MD’,” Jonathan Rochkind, a systems librarian, wrote on his blog Bibliographic Wilderness, adding that research librarians ought to be experts in “evaluating credibility, authenticity, and authority of apparently scholarly literature.”
“It is part of our job to help people look at information and determine if it’s credible or not,” says Lee-Anne Ufholz, a health sciences research liaison at the University of Ottawa.
Although research librarians aren’t experts in medical content, they are experts in finding information and assessing the many markers that help indicate the level of quality of scientific literature. These markers include where the journal is indexed, which libraries hold it and how often it is cited in other journals.
“If the paper trail is weak, maybe the evidence you expect to glean from the article is also weak, so you should look at it very critically,” says Dean Giustini, a biomedical librarian at the University of British Columbia in Vancouver.
The paper trail (perhaps “electronic trail” might be more accurate today) for The Australasian Journal of Bone and Joint Medicine was indeed a faint one. It was only held by one library: the State Library of New South Wales in Sydney, Australia. Odder still, the journal was absent from almost all major academic publishing indices, (except for World-Cat), including PubMed, Ulrich’s Periodicals Directory, Science Direct, Scopus and World of Science.
“If you are a legitimate publisher in the medical field, you want to be indexed,” says Julia Blixrud, assistant executive director of scholarly communication for the Washington, DC-based Association of Research Libraries.
The journal also lacked a dedicated website. The lack of an Internet presence, librarians say, is unheard of for a modern journal and raises suspicions that a publisher is hiding something. Even if a journal has a website, though, it doesn’t mean the publication is credible. Librarians say the website of a journal should list its editorial board, indicate if it is peer reviewed and contain instructions for authors.
“If it doesn’t have any instructions for authors, you have to wonder: ‘Why aren’t you accepting submissions?’” says Ufholz.
By way of example, Ufholz points to the lack of submission instructions on the website for The Journal of Global Drug Policy and Practice (www.globaldrugpolicy.org/index.php). Critics of this journal are plentiful; some claim it arose merely to combat harm-reduction drug policies (which focus on personal choice and safe habits for drug use), and point out that it’s funded by the Office of Juvenile Justice and Delinquency Prevention, which is part of the US Department of Justice. Still, the publication claims to be an “international journal with the goal of bridging the information gap on drug policy issues between the medical-scientific community, policymakers, practitioners and the lay public.”
Marlene Dorgan, president of the Canadian Health Libraries Association, also points to The Journal of Global Drug Policy and Practice as a publication that appears to be driven more by a political agenda than science. “That journal, which looks legitimate, which is being used by the Canadian government to back up various decisions, is supported by groups that believe enforcement is the route to reducing drug use.”
In addition to checking scientific indices and analyzing websites, research librarians look for answers to other questions when assessing the quality of a medical journal. What is the journal’s impact factor? Are prominent researchers contributing articles? Do the same authors appear again and again? Does it have an International Standard Serial Number? Do the articles contain citations mostly to obscure websites instead of recognized journals? Do the authors repeatedly cite previous pieces they have written for the same journal?
Traditionally, the reputation of the publisher has also lent weight to the credibility of a medical journal. That’s less true today, some claim, because the major publishers have become so large that their catalogues are bound to include low-quality and even suspect fare.
“Even with these big-name publishing companies, it’s becoming harder to see if journals are legitimate or not,” says Dorgan, a librarian with the University of Alberta in Edmonton.
Librarians are quick to point out that Elsevier is not the only publishing house which has been accused of sometimes blurring the line between advertising and science. Giustini, who runs a popular wiki for health librarians, says this problem is particularly pronounced in medicine, where there is much money to be made from sales of drugs and medical devices. Drug companies push to publish positive results, he says, because it is an effective means of getting the attention of doctors, who prescribe drugs and recommend medical devices. So it shouldn’t come as a complete surprise, he says, that a drug company (Merck, in the case of the fake Australian journal) funded a marketing publication in a form that looked like a journal. “It’s troubling and shocking but not completely new.”
The effects of polluting medical literature with poor or suspect science are significant, says Giustini. Doctors are too busy to carefully assess every article about a new drug or device, he says, yet must now take that time because they are wary of accepting in good faith the authors’ conclusions.
“This will slow down the whole mechanism of health care,” says Giustini, who adds: “I think that what we want to work towards is ensuring the integrity of the medical bibliography and medical literature.”