Comment

What are the implications of optimism bias in clinical research?

During the first wave of COVID-19, hydroxychloroquine (HCQ) was used off-label despite the absence of evidence documenting its clinical benefits. Since then, a meta-analysis of randomised trials showed that HCQ use was associated with an 11% increase in the mortality rate. We aimed to estimate the number of HCQ-related deaths worldwide.

We estimated the worldwide in-hospital mortality attributable to HCQ use by combining the mortality rate, HCQ exposure, number of hospitalised patients, and the increased relative risk of death with HCQ. The mortality rate in hospitalised patients for each country was calculated using pooled prevalence estimated by a meta-analysis of published cohorts. The HCQ exposure was estimated using median and extreme estimates from the same systematic review. The number of hospitalised patients during the first wave was extracted from dedicated databases. The systematic review included 44 cohort studies (Belgium: k = 1, France: k = 2, Italy: k = 12, Spain: k = 6, Turkey: k = 3, USA: k = 20). HCQ prescription rates varied greatly from one country to another (range 16–84%). Overall, using median estimates of HCQ use in each country, we estimated that 16,990 HCQ-related in-hospital deaths (range 6267–19256) occurred in the countries with available data. The median number of HCQ-related deaths in Belgium, Turkey, France, Italy, Spain, and the USA was 240 (range not estimable), 95 (range 92–128), 199 (range not estimable), 1822 (range 1170–2063), 1895 (range 1475–2094) and 12739 (3244− 15570), respectively.

Although our estimates are limited by their imprecision, these findings illustrate the hazard of drug repurposing with low-level evidence.

Previous studies have found that overestimating treatment effects (i.e. ‘optimism bias’) leads to underpowered clinical trials. The prevalence of ‘optimism bias’ in contemporary National Clinical Trials Network (NCTN) cancer clinical trials is unknown.

We conducted a systematic review of NCTN phase III randomized trials published from January 2007 to January 2017. We compared the hypothesized versus observed treatment effects in each trial, and examined whether trial-related factors were correlated with the study results. We also reviewed the methods of each protocol to assess whether a rationale for the hypothesized effect size was provided.

We identified 161 clinical trials, of which 130 were eligible for analysis. Original protocols could not be located for 8 trials (5.0%). Twenty-eight trials (21.5%) observed a statistically significant difference in the primary end point favoring the experimental treatment. The median ratio of observed-to-expected hazard ratios among trials that observed a statistically significant effect on the primary end point was 1.07 (range: 0.33–1.28) versus 1.32 (range: 0.86–2.02) for trials that did not, compared with 1.34 and 1.86, respectively, for National Cancer Institute (NCI) trials published between 1955 and 2006. An effect size at least as large as the one projected in the protocol trials was observed in 9.8% of trials, compared with 17% of NCI trials published from 1955 to 2006. Most trials (64.6%) provided no rationale to support the magnitude of the proposed treatment effect in the protocol.

Despite a reduction in ‘optimism bias’ compared with previous eras, most contemporary NCTN phase III trials failed to establish statistically significant benefits of new cancer therapies. Better rationalization of proposed effect sizes in research protocols is needed.

The purpose of this 2-arm parallel trial was to assess the effect of the AcceleDent Aura appliance (OrthoAccel Technologies, Houston, Tex) on the rate of maxillary premolar extraction space closure in adolescent patients.

Forty Class II adolescents treated with full fixed appliances and maxillary premolar extractions participated in this randomized clinical trial. They were recruited in a private practice and treated by 1 clinician. Randomization was accomplished in blocks of 10 patients assigned to either a no-appliance group or the AcceleDent Aura appliance group with the allocations concealed in opaque, sealed envelopes. Both the operator and the outcome assessor were blinded; however, it was not feasible to blind the patients. Models were taken of the maxillary arch at the start of space closure and just before complete space closure. The space was measured parallel to the occlusal plane from the cusp tips of the teeth mesial and distal to the extraction spaces.

There was no clinically (0.05 mm per month; 95% confidence interval [CI], –0.24, 0.34) or statistically significant difference in the rate of space closure (P = 0.74). In both the univariable and multivariable analyses, the mean rate of tooth movement was slower by 0.13 mm per month (95% CI, –.26, .005) on the left side compared with the right side, but this was not statistically significant (P = 0.06).

The AcceleDent Aura appliance had no effect on the rate of maxillary premolar extraction space closure. Only a few participants were considered to be good compliers with the appliance. However, the rate of space closure in the good compliers was similar to the overall group and did not appear to influence the result.

This trial was not registered.

The protocol was not published before trial commencement.