Do clinicians understand the size of treatment effects? A randomized survey across 8 countries
Abstract
Background: Meta-analyses of continuous outcomes typically provide enough information for decision-makers to evaluate the extent to which chance can explain apparent differences between interventions. The interpretation of the magnitude of these differences — from trivial to large — can, however, be challenging. We investigated clinicians’ understanding and perceptions of usefulness of 6 statistical formats for presenting continuous outcomes from meta-analyses (standardized mean difference, minimal important difference units, mean difference in natural units, ratio of means, relative risk and risk difference).
Methods: We invited 610 staff and trainees in internal medicine and family medicine programs in 8 countries to participate. Paper-based, self-administered questionnaires presented summary estimates of hypothetical interventions versus placebo for chronic pain. The estimates showed either a small or a large effect for each of the 6 statistical formats for presenting continuous outcomes. Questions addressed participants’ understanding of the magnitude of treatment effects and their perception of the usefulness of the presentation format. We randomly assigned participants 1 of 4 versions of the questionnaire, each with a different effect size (large or small) and presentation order for the 6 formats (1 to 6, or 6 to 1).
Results: Overall, 531 (87.0%) of the clinicians responded. Respondents best understood risk difference, followed by relative risk and ratio of means. Similarly, they perceived the dichotomous presentation of continuous outcomes (relative risk and risk difference) to be most useful. Presenting results as a standardized mean difference, the longest standing and most widely used approach, was poorly understood and perceived as least useful.
Interpretation: None of the presentation formats were well understood or perceived as extremely useful. Clinicians best understood the dichotomous presentations of continuous outcomes and perceived them to be the most useful. Further initiatives to help clinicians better grasp the magnitude of the treatment effect are needed.
- Accepted September 9, 2015.
In this issue
Article tools
Jump to section
Related Articles
Cited By...
- Understanding effect size: an international online survey among psychiatrists, psychologists, physicians from other medical specialities, dentists and other health professionals
- Evidence-Based Practice Competencies among Nutrition Professionals and Students: A Systematic Review
- Management of chronic pain secondary to temporomandibular disorders: a systematic review and network meta-analysis of randomised trials
- Going against the status quo in screening: Call to action to improve teaching in preventive health care
- A lencontre du statu quo en matiere de depistage: Appel a laction pour ameliorer lenseignement des soins de sante preventifs
- Updating the Canadian clinical practice guideline for managing pediatric obesity: a protocol
- Understanding of research results, evidence summaries and their applicability--not critical appraisal--are core skills of medical curriculum
- Efficacy and safety of low and very low carbohydrate diets for type 2 diabetes remission: systematic review and meta-analysis of published and unpublished randomized trial data
- Artificial Intelligence in Cardiology
- Understanding and communicating risk: Measures of outcome and the magnitude of benefits and harms
- Comprendre et expliquer le risque: Mesurer les resultats et lampleur des bienfaits et des prejudices
- Presentation approaches for enhancing interpretability of patient-reported outcomes (PROs) in meta-analysis: a protocol for a systematic survey of Cochrane reviews
- Prendre de meilleures decisions en matiere de depistage preventif: Equilibrer bienfaits et prejudices
- Better decision making in preventive health screening: Balancing benefits and harms
- Multilayered and digitally structured presentation formats of trustworthy recommendations: a combined survey and randomised trial
- Minimal important difference is important