Sample sizes in COVID-19–related research

Cheung and colleagues warned that underpowered studies that committed a type II error will discourage clinicians from using effective treatment.1 I agreed with this argument. Because the number of published clinical trials on coronavirus disease 2019 (COVID-19) patients has been increasing rapidly, I have reviewed all these trials published between Jan. 1, 2020, and Mar. 25, 2020, and indexed in PubMed, and assessed the quality of their sample size calculation.

I identified a total of 374 articles, 4 of which described trials. In general, the quality of sample size calculation was not acceptable. One study did not justify the sample size.2 One assumed that the treatment can reduce the outcome variable by 40%, but the Cohen’s d effect size should have been provided instead.3 One study did not explicitly state the nonzero assumption of the control group effect4 (they assumed the effect of the control group would be about 5%, according to the Fleiss formula with continuity correction used by the authors5). The fourth study did provide the effect size estimation, but the sample size calculated in the paper deviated from that calculated using the standard formula by 6% (the percentage of patients reaching the outcome within the study period should be 71.1%, as calculated according to the assumptions given by the authors, but in the article, they overestimated that to be 75%).6 The power of their sample size would be 78% instead of the desired 80%.

Given the unacceptable quality of the sample size calculation of COVID-19 trials, I strongly suggest that all research teams include a statistician or invite a statistician to evaluate the appropriateness of the sample size calculation.