Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal antiinflammatory drugs in rheumatoid arthritis

https://doi.org/10.1016/0197-2456(89)90017-2Get rights and content

Abstract

Important design aspects were decreasingly reported in NSAID trials over the years, whereas the quality of statistical analysis improved. In half of the trials, the effect variables in the methods and results sections were not the same, and the interpretation of the erythrocyte sedimentation rate in the reports seemed to depend on whether a significant difference was found.

Statistically significant results appeared in 93 reports (47%). In 73 trials they favored only the new drug, and in 8 only the active control. All 39 trials with a significant difference in side effects favored the new drug. Choice of dose, multiple comparisons, wrong calculation, subgroup and within-groups analyses, wrong sampling units (in 63% of trials for effect variables, in 23% for side effects), change in measurement scale before analysis, baseline difference, and selective reporting of significant results were some of the verified or possible causes for the large proportion of results that favored the new drug.

Doubtful or invalid statements were found in 76% of the conclusions or abstracts. Bias consistently favored the new drug in 81 trials, and the control in only one trial.

It is not obvious how a reliable meta-analysis could be done in these trials.

References (114)

  • PC Gøtzsche

    Reference bias in reports of drug trials

    Br Med J

    (1987)
  • D Mainland

    Elementary Medical Statistics

    (1963)
  • F. Galton

    Regression towards mediocrity in hereditary stature

    J Anthropol Inst Great Britain Ireland

    (1885–1886)
  • RR Sokal et al.

    Biometry

    (1981)
  • M. Hollander et al.

    Non-parametric Statistical Methods

    (1973)
  • SJ Pocock

    Clinical trials. A practical approach

    (1983)
  • Anonymous: Diagnostic criteria for rheumatoid arthritis. 1958 revision

    Ann Rheum Dis

    (1959)
  • O. Steinbrocker et al.

    Therapeutic criteria in rheumatoid arthritis

    JAMA

    (1949)
  • PC Gøtzsche et al.

    Rational choice of dose: insufficient background knowledge?

    Ration Drug Ther

    (1986)
  • DM Ritchie et al.

    Clinical studies with an articular index for the assessment of joint tenderness in patients with rheumatoid arthritis

    Q J Med

    (1968)
  • WH Cochran et al.

    Experimental Designs

    (1957)
  • JE Grizzle

    The two-period change-over design and its use in clinical trials

  • GG Koch

    The use of non-parametric methods in the statistical analysis of the two-period change-over design

    Biometrics

    (1972)
  • M Hills et al.

    The two-period cross-over clinical trial

    Br J Clin Pharmacol

    (1979)
  • P Armitage

    Statistical Methods in Medical Research

    (1971)
  • AR Feinstein

    An additional basic science for clinical medicine: II. The limitations of randomized trials

    Ann Intern Med

    (1983)
  • R Peto et al.

    Design and analysis of randomized clinical trials requiring prolonged observation of each patient. I. Introduction and design

    Br J Cancer

    (1976)
  • JW Tukey

    Some thoughts on clinical trials, especially problems of multiplicity

    Science

    (1977)
  • K Godfrey

    Comparing the means of several groups

    N Engl J Med

    (1985)
  • PD Oldham

    Measurement in Medicine: The Interpretation of Numerical Data

    (1968)
  • L Lasagna

    The clinical measurement of pain

    Ann New York Acad Sci

    (1960)
  • D Mainland

    The estimation of inflammatory activity in rheumatoid arthritis. Role of composite indices

    Arthr Rheum

    (1967)
  • E Juhl et al.

    The epidemiology of the gastrointestinal randomized clinical trial

    N Engl J Med

    (1977)
  • E Hemminki

    Quality of reports of clinical trials submitted by the drug industry to the Finnish and Swedish control authorities

    Eur J Clin Pharmacol

    (1981)
  • RH Fletcher et al.

    Clinical research in general medical journals

    N Engl J Med

    (1979)
  • A Ambroz et al.

    Deficiencies of randomized control trials

    Clin Res

    (1978)
  • R DerSimonian et al.

    Reporting on methods in clinical trials

    N Engl J Med

    (1982)
  • JD Emerson et al.

    Reporting clinical trials in general surgical journals

    Surgery

    (1984)
  • JM Bland et al.

    Is the clinical trial evidence about new drugs statistically adequate?

    Br J Clin Pharmacol

    (1985)
  • MJ Gardner et al.

    Use of check lists in assessing the statistical content of medical studies

    Br Med J

    (1986)
  • DL Scott et al.

    Variations in response to non-steroidal, anti-inflammatory drugs

    Br J Clin Pharmacol

    (1982)
  • L Lasagna

    Problems in publication of clinical trial methodology

    Clin Pharmacol Ther

    (1979)
  • PC Gøtzsche

    Enalapril, atenolol, and hydrochlorothiazide in hypertension (letter)

    Lancet

    (1986)
  • TR Karlowski et al.

    Ascorbic acid for the common cold. A prophylactic and therapeutic trial

    JAMA

    (1975)
  • EC Huskisson et al.

    How blind is double blind? And does it matter?

    Br J Clin Pharmacol

    (1976)
  • P Lee et al.

    The evaluation of antirheumatic drugs

    Curr Med Res Opin

    (1973)
  • D Schwartz et al.

    Clinical Trials

    (1980)
  • Cooperating Clinics Committee of the American Rheumatism Association

    A seven-day variability study of 499 patients with peripheral rheumatoid arthritis

    Arthr Rheum

    (1965)
  • JR Kirwan et al.

    Clinical judgement analysis—practical application in rheumatoid arthritis

    Br J Rheumatol

    (1983)
  • Cited by (288)

    • Bias Assessment in Outcomes Research: The Role of Relative Versus Absolute Approaches

      2021, Value in Health
      Citation Excerpt :

      For the quality effects model, bias adjustment was achieved using additional data in the model (relative ranks) apart from the usual effect size and standard error.22,23 We examined the same 25 quality assessment scales18,24-47 to keep consistency with those reported previously3,17 and used their most recent version if available47 (Appendix Table 2 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.02.011). All distinct bias safeguard items from the 25 scales were examined and trimmed of items deemed to be reporting quality only (see Appendix Table 3 in Supplemental Materials found at https://doi.org/10.1016/j.jval.2021.02.011 for final list).

    • Management of Rheumatoid Arthritis and the Pharmacist's Role

      2019, Encyclopedia of Pharmacy Practice and Clinical Pharmacy: Volumes 1-3
    • Management of rheumatoid arthritis and the pharmacist’s role

      2019, Encyclopedia of Pharmacy Practice and Clinical Pharmacy
    View all citing articles on Scopus
    View full text