Shedding light on retractions

If one were inclined, for some reason, to strike fear into the heart of a medical researcher, it would take little more than whispering a single word: retraction. The phrase “grant application denied” isn’t too popular in academic circles, either, but seeing that in a letter is unlikely to distress researchers as much as seeing “retraction” stamped on studies bearing their names.

When a medical journal retracts a study, there are many losers and few, if any, winners. The study’s authors add blemishes to their resumes. The journal also looks bad, forced to admit it allowed a wonky paper into its pages.

You might think the scientific community benefits when a flawed study is flagged. And don’t patients, who might suffer harm from faulty treatments derived from faulty data, also win when an invalid paper is retracted? In theory, yes, but in reality, no, as neither science nor the public gains much, because medical journals generally handle retractions poorly.

“Most journals don’t do a good job of either explaining why they retracted a paper, or of publicizing the retraction,” says Dr. Ivan Oransky, executive editor of Reuters Health and cofounder, with fellow United States medical journalist Adam Marcus, of the blog Retraction Watch (http://retractionwatch.wordpress.com).

The purpose of the blog, launched in August 2010, is to peek into the dark corners where journals stash their retractions and drag that information out into the light. The blog has been a hit, receiving praise from many in the scientific community, including Dr. Ben Goldacre, author of the “Bad Science” column in London, United Kingdom-based newspaper The Guardian, who recommended that people read Retraction Watch because “eyeballs are an excellent disinfectant” for contaminated research.

Oransky and Marcus seek out retractions on their own and also receive tips from readers. When writing about a retraction, they will attempt to learn details about the study’s flaws by contacting one of its authors and the editor of the journal that published it. The responses aren’t always positive.

Many journal editors appear reluctant to reveal the reasons for retraction of an article.

Image courtesy of © 2011 Jupiterimages Corp.

When reporting on the retraction of a study in the Annals of Thoracic Surgery, for example, Marcus contacted the journal’s editor, Dr. Henry Edmunds. The language of the retraction was vague, which isn’t uncommon, and Marcus asked Edmunds to clarify why the study was retracted. There was nothing vague about Edmunds response to Retraction Watch: “That’s none of your damn business” (http://retractionwatch.wordpress.com/2011/01/05/why-was-that-paper-retracted-editor-to-retraction-watch-its-none-of-your-damn-business).

Most editors aren’t quite as blunt, though many are reluctant to shed light on the reason for a retraction. “Many journal publishers seem to believe we don’t need that information,” says Oransky. “We find that insulting to science.”

Of course, not all journals shy away from explaining retractions. In February, for example, the journal Anesthesia & Analgesia published a lengthy letter describing why 22 papers were retracted (www.anesthesia-analgesia.org/site/misc/25.February.2011.Notice.pdf). According to Dr. Steve Shafer, the journal’s editor, a retracted study should not be expunged because that would “burn a hole in the fabric of scientific knowledge,” leading only to further damage. “The retraction notice becomes part of the fabric as well,” Shafer writes in an email. “The retraction notice must include sufficient information to permit readers to accurately infer exactly why the article has been retracted, and, ideally, how that retraction affects our knowledge.”

By contrast, there are retraction notices that tell the reader almost nothing, such as the two recently announced by the Journal of Biological Chemistry, which both read: “This manuscript has been withdrawn at the request of the authors” (www.jbc.org/content/285/42/32678.1.full, www.jbc.org/content/285/42/32678.2.full). Why did the authors request the retractions?

“That is considered confidential information,” Nancy Rodnan, director of publications for the American Society for Biochemistry and Molecular Biology, writes in an email.

According to Dr. Ferric Fang, editor-in-chief of Infection and Immunity, retractions are serious matters and his journal does not take them lightly. “The process for determining whether an article should be retracted and whether sanctions are warranted is an involved one, as it must be, to insure fairness to both the authors and the readership,” Fang writes in an email. “Retraction statements are carefully worded with an intent to be clear, accurate and fair.”

Not only should retractions be clear, some researchers say, they should also be published sooner to the date when editors become certain of a study’s flaws. Far too often, retractions are issued long after any doubts about a study’s problems have been put to rest. For example, the infamous paper by Dr. Andrew Wakefield that linked autism to vaccines has frequently been cited as the epitome of the should-have-been-retracted-long-ago study. The Lancet, which published the paper in 1998, did not issue a retraction until Feb. 2, 2010 — prompting many responses similar to the headline of a BMJ article: “Why did The Lancet take so long?” (BMJ 2010;340:c644).

Though often handled poorly and slowly when they do occur, retractions are rare in medical journals. When Barbara Redman, a bioethicist and dean of the Wayne State University College of Nursing in Detroit, Michigan, looked at more than five million records in the PubMed database from the period 1995–2004, she found that only 328, or 0.0065%, were retracted (J Med Ethics 2008;34:807-9).

Perhaps the most troubling finding was the frequency with which retracted studies, which are not expunged from databases or journal archives, are cited in scientific literature after their retraction dates. The 315 retracted papers from English journals were cited 3942 times before retraction and 4501 times after retraction. Researchers often don’t check their citations to see if they are still valid, says Redman, which undermines the purpose of retractions: to cleanse scientific literature of its offal.

“People will continue to cite the articles as if there were as good as gold,” says Redman.

Unless researchers take it upon themselves to double-check all their citations, or are compelled to do so by journals as a condition of publication, retracted studies will continue to propagate through scientific literature. “The bottom line is that the system of retraction does not work,” says Redman.

Another major problem with the system is that all retracted papers are lumped into one category, even though papers are retracted for numerous reasons, from innocent mistakes to outright fraud. “Retraction is a very blunt instrument and it’s not applied fairly,” says Grant Steen, president of Medical Communications Consultants in Chapel Hill, North Carolina.

When Steen examined 742 retracted papers, he discovered that there are eight different reasons a paper could be retracted (J Med Ethics doi:10.1136/jme.2010.040923). Most retractions, 73.5%, fall into the broad category of error, which consists of scientific mistakes (31.5%), duplicate publications (15.8%), plagiarism (14.4%), ethical violations (10.2%), unstated reasons (8.2%) and journal error (3.6%). The remaining 26.6% of retractions come under the banner of fraud, either in the form of data fabrication (15%) or data falsification (13.2%).

Like others who have studied retractions, Steen found that journals are rarely clear in their explanations of why retractions were necessary and do a poor job of identifying retracted papers. Of the 742 studies he examined, 41.1% were identified as retracted using watermarks on the pdf versions, and 17.3% alerted readers via notes appended to the pdfs. A third of the studies were noted as retracted on the journals’ websites (including almost half of those already watermarked). A large portion of the retracted papers in his study, 31.8%, were not identified as having been retracted at all.

“From the journal’s standpoint, they should be crystal clear about why something is retracted, but usually they either say nothing or say something ambiguous,” says Steen. “If there is a scientific error, they should explain that error in gory detail.”