Short communicationReporting of treatment effects from randomized trials: A plea for multivariable risk ratios
Section snippets
Background
Randomized trials are considered the gold standard to study effects of medical interventions, primarily because randomization (theoretically) results in balanced distributions of confounders between the study groups. In reality, however, some of these confounders may not be well-balanced, resulting from flaws in the randomisation procedure or, more often, because of ‘bad luck’ [1], [2]. Then, the observed effect may not be (completely) attributable to the treatment under study, and adjustment
Clinical examples
In a randomized trial on the effect of tissue plasminogen activator vs. streptokinase thrombolytic therapy on mortality in patients with myocardial infarction, the crude odds ratio (OR) was 0.85 (95% CI 0.78–0.94; p-value = 0.0012) [6]. Adjustment for 17 well balanced baseline characteristics resulted in an OR of 0.82 (95% CI 0.74–0.91; p-value = 0.0002). In another randomized trial on insulin therapy in very-low-birth-weight infants the difference between crude and adjusted OR was more pronounced
Non-collapsibility of the odds ratio
Crude effect estimates are also called marginal, or collapsed, effects, and adjusted effect estimates are called conditional effects. If potential confounders are balanced between treatment groups in a randomized trial, adjustment for these characteristics is expected not to result in a change in the effect estimate, i.e. the marginal effect equals the conditional effect.
Indeed, when adjusting for balanced covariates, the marginal and conditional effect estimates are the same, when applying for
Precision
In linear models, adjustment for balanced covariates will not affect the estimate of the treatment effect, but will increase precision (smaller standard errors, confidence intervals and p-values). In logistic models, however, the effect estimate increases when adjusting for balanced covariates (moves away from the null effect), while the standard errors increase [6], [7]. Altogether, the change in the effect estimate is larger than the increase in the standard error of the estimate and as a
Personalized treatment effects
If the treatment effect is the same among men and women (i.e., no effect modification), the conditional effect is a more individualised effect, whereas the marginal effect is rather a population averaged effect. Obviously, a patient is either a man or a woman and not an average of the two sexes. Hence, the effect a patient will experience from a treatment is the effect as observed in either men or women. Thus, the effect an individual patient will experience is the conditional effect, not the
Odds ratio or risk ratio?
One of the advantages of odds ratios over risk ratios, is that odds ratios are symmetric with respect to ‘successes’ and ‘failures’, meaning for example that the odds ratio for mortality is exactly the reciprocal of the odds ratio for survival [14]. A disadvantage of odds ratios, as illustrated above, is the phenomenon of non-collapsibility, which may cause the marginal and the conditional treatment effects to differ considerably. Another feature of the odds ratio is that it is often
Conclusions
Clinicians are typically interested in the effects of medical interventions that apply to individual patients, which are conditional effects rather than marginal effects. However, when considering odds ratios, even in a perfectly balanced trial, conditional and marginal effects may differ due to non-collapsibility of the odds ratio. If covariate adjustment is indicated (for example to adjust for confounding) the situation becomes even more complicated, since it will be hard to distinguish
Competing Interests
All authors declare that they have no competing interests.
References (20)
- et al.
Clinical trials in acute myocardial infarction: should we adjust for baseline characteristics?
Am Heart J
(2000) - et al.
Covariate adjustment in randomized controlled trials with dichotomous outcomes increases statistical power and reduces sample size requirements
J Clin Epidemiol
(2004) Clinical trials in acute myocardial infarction: when should we adjust for baseline characteristics?
Am Heart J
(2000)Quantifying the magnitude of baseline covariate imbalance resulting from selection bias in randomized trials
Biom J
(2005)- et al.
Understanding controlled trials: baseline imbalance in randomized controlled trials
BMJ
(1999) Testing for baseline balance in clinical trials
Stat Med
(1994)Statistical principles for clinical trials
Stat Med
(1999)- et al.
Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems
Stat Med
(2002) - et al.
Early insulin therapy in very-low-birth-weight infants
N Engl J Med
(2008) - et al.
Biased estimates of treatment effect in randomized experiments with nonlinear regressions and omitted covariates
Biometrika
(1984)
Cited by (23)
Clinical research study implementation of case-finding strategies for heart failure and chronic obstructive pulmonary disease in the elderly with reduced exercise tolerance or dyspnea: A cluster randomized trial
2020, American Heart JournalCitation Excerpt :Comparison of the continuous outcomes (EQ-5D, SF-36 scales, KATZ-15) between the 2 strategies was done using a linear mixed model. The analysis included baseline measurements of the continuous outcomes as a covariate and a random intercept to correct for clustering of patients in practices and expressed as adjusted differences in means with 95% CIs.23 Comparison of the dichotomous outcomes, that is, newly detected diagnoses (primary outcome) and health care use, was analyzed applying a log binomial mixed model and expressed as adjusted relative risk with 95% CIs.23,24
Effects of nursery songs for infants and young children's anxiety: A reporting mistake of effect measure of association
2016, American Journal of Emergency MedicineDifferences in interaction and subgroup-specific effects were observed between randomized and nonrandomized studies in three empirical examples
2013, Journal of Clinical EpidemiologyCitation Excerpt :However, today’s clinical practice is shifting more and more toward individually tailored care. Personalized care requires knowledge on the effects of medical interventions at an individual rather than at a patient population level [1]. Compared with main effect estimates, subgroup analyses move toward a more personalized estimate.
Covariate adjustment increased power in randomized controlled trials: An example in traumatic brain injury
2012, Journal of Clinical EpidemiologyCitation Excerpt :In contrast to effects measured by the unadjusted OR for the reference model, covariate adjustment yields adjusted OR that should be interpreted conditionally on the covariates included in the model rather than at a population level. This is in contrast to adjustment in linear regression modeling and binomial regression where adjusted treatment effects have the same population-level interpretation as unadjusted treatment effects [24,25]. For example, adjustment for age would produce OR for a specific age.
Overestimation of risk ratios by odds ratios in trials and cohort studies: Alternatives to logistic regression
2012, CMAJ. Canadian Medical Association JournalCitation Excerpt :Therefore, any misinterpretation of odds ratios should be avoided with calculation and presentation of adjusted risk ratios in both cohort studies and RCTs. Also, if adjustment for baseline covariates is not done, which is often the case in RCTs, the risk ratio is the preferred measure of association in case of dichotomous outcomes.21 Note that in case–control studies, the odds ratio is the appropriate effect estimate and the odds ratio can be interpreted as a risk ratio or rate ratio depending on the sampling method.1–4