Interpreting the results of small trials
========================================

* Frank Molnar
* Malcolm Man-Son-Hing
* Anna Byszewski
* Nahid Azid

The randomized controlled trial of preventive home nursing visits for frail elderly people reported by Dawn Dalby and colleagues1 raised several key issues regarding the design, interpretation and reporting of trials testing the efficacy of interventions in the elderly population.

With the testing of complicated interventions and the chronic shortage of resources in this area, clinical trials may be conducted with inadequate numbers of patients to reliably demonstrate positive effects. Dalby and colleagues enrolled 142 patients, giving the trial a prestudy power of 50%. Inadequate sample size will likely result in findings that have high statistical variability (low precision). Consequently, the 95% confidence intervals around the point estimates of the primary outcome for the control and intervention groups will be imprecise and are likely to overlap and result in statistically insignificant results. As the number of patients in the trial increases, the 95% confidence intervals become more precise with less overlap (if there is a positive treatment effect) and the results may become statistically significant. For these reasons, the study by Dalby and colleagues does not demonstrate that nursing visits are ineffective. In fact, no firm conclusions can be drawn from its results.

The authors cited their lack of adequate power as a possible explanation for their lack of statistically significant results. From a methodological and theoretical perspective, Goodman and Berlin2 have argued against the use of post hoc power to explain negative trials. Once a trial is completed, they argue, the use of confidence intervals, rather the post hoc power, is the proper way to interpret trials with results that do not reach statistical significance.

The danger inherent in conducting small, inadequately powered trials is that potentially effective interventions will be judged as ineffective simply because of the inability to detect statistically significant and clinically important benefits. Consequently, caution must be exercised when embarking on small trials and interpreting the results.

## References

1.  1. Dalby DM, Sellors JW, Fraser FD, Fraser C, van Ineveld C, Howard M. Effect of preventive home visits by a nurse on the outcomes of frail elderly people in the community: a randomized controlled trial. CMAJ 2000;162(4):497-500.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxNjIvNC80OTciO3M6NDoiYXRvbSI7czoyNDoiL2NtYWovMTYyLzEwLzE0MDEuMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

2.  2. Goodman SN, Berlin JA. The use of predicted confidence intervals when planning experiments and the misuse of power when interpreting results. Ann Intern Med 1994;121:200-6.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.7326/0003-4819-121-3-199408010-00008&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=8017747&link_type=MED&atom=%2Fcmaj%2F162%2F10%2F1401.1.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1994NY33800008&link_type=ISI)