Commentary Editorial comment
Simple solution to a common statistical problem: Interpreting multiple tests

https://doi.org/10.1016/S0149-2918(04)90078-1Get rights and content

Abstract

Background: The misinterpretation of the results of multiple statistical tests is an error commonly made in scientific literature. When testing several outcome variables simultaneously, many researchers declare a statistically significant result for each test having a P value of <0.05, for example. This approach ignores the fact that, based on a probability result called the Bonferroni inequality, the risk of incorrectly declaring as significant ≥1 test result increases with the number of tests conducted. The implication of this practice is that many scientific results are presented as statistically significant when the underlying data do not adequately support such a claim (sometimes referred to as false-positive results). Although the sequentially rejective Bonferroni test is well known among statisticians, it is not used routinely in scientific literature.

Objective: The intent of this article was to increase the awareness and understanding of the sequentially rejective Bonferroni test, thereby expanding its use.

Methods: This article describes the statistical problem and demonstrates how the use of the sequentially rejective Bonferroni test ensures that incorrect declarations of statistical significance for ≥1 test result are bounded by 0.05, for example.

Conclusion: The sequentially rejective Bonferroni test is an easily applied, versatile statistical tool that enables researchers to make simultaneous inferences from their data without risking an unacceptably high overall type I error rate.

References (8)

  • H.J Khamis

    Deciding on the correct statistical technique

    J Diagn Med Sonography

    (1992)
  • D Wackerly et al.

    Mathematical Statistics with Applications

  • D.F Morrison

    Multivariate Statistical Methods

  • J.L Fleiss

    The Design and Analysis of Clinical Experiments

There are more references available in the full text version of this article.

Cited by (77)

  • Use of antibiotics during pregnancy and risk of spontaneous abortion

    2017, CMAJ
    Citation Excerpt :

    Fourth, given that many early pregnancy losses are not recognized clinically, we excluded spontaneous abortions that occurred before 6 completed weeks of gestation. Fifth, given that we performed 27 hypothesis tests in our study, we adjusted for multiple testing using the Bonferroni correction.20 We considered an association between antibiotic class or type and spontaneous abortion to be statistically significant if the p value was less than 0.002 (i.e., 0.05/27).

View all citing articles on Scopus

Dr. Gordi is now with the Department of Pharmaceutical Sciences, State University at Buffalo, Buffalo, New York.

Dr. Khamis is now with the Statistical Consulting Center, Wright State University, Dayton, Ohio.

View full text