Original articles
Discrepancy between published report and actual conduct of randomized clinical trials

https://doi.org/10.1016/S0895-4356(02)00440-7Get rights and content

Abstract

Randomized clinical trial (RCT) publications with inappropriate random-sequence generation, lack of allocation concealment, or imperfect blinding yield inflated estimates of effect compared to those in which adequate methods are described. RCTs that do not state methods used yield similar effect estimates, suggesting that inadequate methods were used. We compared RCT publications with investigator reports of actual practice for 40 rheumatology RCTs published in 1997/1998. In RCTs in which these methods were not described in the trial reports and would thus have been characterized as “inadequate,” investigators reported using methods of random-sequence generation and allocation concealment that would be considered adequate in 77.4 and 78.1% of RCTs, respectively. This suggests that, in contrast to previous reports, inadequate random-sequence generation and allocation concealment, per se, may not be a major problem in RCTs. Characterizing RCTs as “good” or “poor” quality based on the published report is likely to be inappropriate.

Introduction

Increasingly, authors of meta-analyses are assessing the methodology of randomized clinical trials (RCT) as a means of determining the “quality” of RCTs. These quality scales generally rely on the published account of the RCT. Previous studies have shown that RCT publications that describe inadequate methods of random-sequence generation, allocation concealment, or blinding yield inflated estimates of a treatment's effect, compared to those that describe adequate trial methods 1, 2, 3. In addition, trial publications that do not include any description of allocation concealment yield similar inflated estimates of effect [2]. Allocation concealment is a process distinct from blinding that conceals the random assignment sequence from the investigator and participant before and until allocation to therapy, with the aim of reducing selection bias [4]. The objective of our study was to compare RCT publications with an investigator survey of actual trial conduct for the methods of random-sequence generation, allocation concealment, and blinding.

Section snippets

Methods

As part of a study evaluating the methodology of rheumatology RCTs, we searched for publications in English of RCTs for treatment of rheumatic diseases published in 1997/1998 using MEDLINE and a hand search of rheumatology journals, Annals of the Rheumatic Diseases, Arthritis & Rheumatism, British Journal of Rheumatology, Journal of Rheumatology, Osteoarthritis & Cartilage, and Scandinavian Journal of Rheumatology [5]. Each of the publications was assessed for trial methods by one reviewer

Results

Of the 50 investigators contacted, 40 (80%) responded. There were no significant differences between the published description of adequate random-sequence generation, allocation concealment, and blinding in the RCT publication between survey responders and nonresponders. In the survey, investigators responded that methods of random-sequence generation and allocation concealment that were defined as adequate were actually performed much more frequently than was evident from the RCT publication

Discussion

Our survey results suggest that absence of reporting of methods in a trial report does not necessarily mean that trialists did not carry out these methods. Despite the CONSORT statement (1996) to encourage improved standards for RCT reporting [6], our survey suggests that RCT publications do not contain comprehensive descriptions of the trial methods that are actually used. Our results have implications for the use of quality scoring of RCTs for meta-analytic purposes, in that scales that

Acknowledgements

This work was supported by NIH AR47785. Catherine Hill was supported by a grant from the Arthritis Foundation of Australia.

References (14)

There are more references available in the full text version of this article.

Cited by (94)

  • Unreported formal assessment of unblinding occurred in 4 of 10 randomized clinical trials, unreported loss of blinding in 1 of 10 trials

    2017, Journal of Clinical Epidemiology
    Citation Excerpt :

    All 24 trials that reported assessment of the risk of unblinding had subjective outcomes [16]. It has been repeatedly documented that methodological aspects of randomized clinical trial often are reported inadequately in trial publications [15,21,22,25–27]. This also includes the reporting of blinding procedures [15,21].

View all citing articles on Scopus

Current address: Rheumatology Unit, the Queen Elizabeth Hospital, Woodville, South Australia.

View full text