Essay

Redundancy, disaggregation, and the integrity of medical research

The objective of the study was to estimate the proportion of secondary publications of randomized controlled trials (RCTs) that provide new results relative to the primary publication.

We searched for RCTs published in 2014 in the five medical journals with the highest impact factors. Secondary publications for each primary publication were then identified by their registration number. The main outcome measure was the proportion of secondary publications providing results already reported in the primary publication and/or nonprespecified analyses and/or a meta-analysis pooling results of studies not identified by systematic review.

A total of 144 primary publications were identified; 94 (65%) had at least one secondary publication within 30 months after a primary publication. Of the secondary publications, 20% reported only results present in the primary publication, and 35% reported results not prespecified or pooled analyses not based on a systematic review. Factors associated with having at least one secondary publication were a large number of randomized trial participants (odds ratio [95% confidence interval]: 3.2 [1.1–9.3] for trials with >1,000 vs. ≤500 participants), investigating a biologic product (4.8 [1.4‒16.3] vs. a nonbiologic product) and cardiologic field vs. other fields (7.6 [1.46–39.8]).

Most drug RCTs with results published in high-impact-factor journals had secondary publications. More than half of these secondary publications provided results already reported in the primary publication or results of nonprespecified analyses.

This paper contributes to the ongoing debate on the reliability of published research. In particular, this study focuses on the selective reporting of research findings in clinical trials, defined as the publication of only part of the findings originally recorded during a research study, on the basis of the results. Selective reporting can lead to concerns ranging from publishing flawed scientific knowledge, to skewing medical evidence, to wasting time and resources invested in the conduct of research. Drawing upon a unique hand-collected dataset, this study investigates the contextual factors associated with selective reporting. Using ‘risk of bias’ ratings assessed based on expert judgment and presented in systematic reviews of clinical literature, this study explores whether selective reporting is associated with: (1) the source of institutional support; and, (2) the type of innovation evaluated. The results indicate that the odds of selective reporting are higher for industry-funded studies than for publicly-funded studies; however, this effect is restricted to studies where at least one author is industry-affiliated. In addition, the results suggest that selective reporting is more likely in projects exploring radical innovation, compared to those investigating incremental innovation.