Overestimation of risk ratios by odds ratios in trials and cohort studies: alternatives to logistic regression

Misuse of odds ratios in cohort studies and RCTs

An odds ratio is calculated as the ratio of the odds of the outcome in the patients with the treatment or exposure and the odds of the outcome in the patients without the treatment or exposure. The risk ratio, also referred to as the relative risk, is calculated as the ratio of the risk of the outcome in these two groups. In this article, we illustrate, by means of two empirical examples, that use of odds ratios in cohort studies and RCTs can lead to misinterpretation of results.

Alternatives to logistic regression to estimate adjusted risk ratios

We found eight methods to estimate adjusted risk ratios in the literature (Table 3 5^,7^–9^,14^–19). The Mantel–Haenszel risk ratio method is straightforward and gives a weighted risk ratio over strata of covariables.14^,15 This method is only practicable if adjusting for a small number of categorical covariables (i.e., continuous covariables first need to be categorized). Log–binomial and Poisson regression are generalized linear models that directly estimate risk ratios.7^,8 The default standard errors obtained by Poisson regression are typically too large; therefore, calculation of robust standard errors for Poisson regression may be needed to obtain a correct confidence interval around the risk ratio.9 The other four methods use odds ratios or logistic regression to estimate risk ratios. The Zhang and Yu method is a simple formula that calculates the risk ratio based on the odds ratio and the incidence of the outcome in the unexposed group.5 The doubling-of-cases method concerns changing the data set in such a way that logistic regression yields a risk ratio instead of an odds ratio.17 Again, calculation of robust standard errors may be needed to obtain a correct confidence interval around the risk ratio.18 Lastly, the method proposed by Austin uses the predicted probabilities obtained from a logistic regression model to estimate risk ratios.19 A recent review article of methods to estimate risk ratios and risk differences in cohort studies illustrated several of these eight methods using empirical data.20

Recommendations for clinical researchers

We showed in the clinical examples and simulations that an odds ratio can substantially overestimate the risk ratio. In fact, both are correct, but when an odds ratio is interpreted as a risk ratio, serious misinterpretation with potential consequences for treatment decisions and policy-making can occur, as illustrated by the two clinical examples. Therefore, any misinterpretation of odds ratios should be avoided with calculation and presentation of adjusted risk ratios in both cohort studies and RCTs. Also, if adjustment for baseline covariates is not done, which is often the case in RCTs, the risk ratio is the preferred measure of association in case of dichotomous outcomes.21 Note that in case–control studies, the odds ratio is the appropriate effect estimate and the odds ratio can be interpreted as a risk ratio or rate ratio depending on the sampling method.1^–4 Of course, if data of cohort studies or RCTs are collected so that a time-dependent analysis is possible, Cox regression yielding hazard ratios is recommended because it estimates relative hazards and does not involve problems related to odds ratios.

There are several valid methods to estimate adjusted risk ratios. In a situation with only one or two categorical covariables, for example, to take into account stratified randomization in an RCT (example 2), we recommend use of the simple Mantel–Haenszel risk ratio method. This method can be easily applied by using Rothman’s spreadsheet Episheet (can be downloaded from http://krothman.byethost2.com/). In a situation with more covariables or continuous covariables, we recommend use of log–binomial regression. If log–binomial regression does not converge, Poisson regression with robust standard errors can be applied. Both methods are easy to perform in standard statistical software packages, including SAS, Stata, R and SPSS22^,23 (see Appendix 2, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.101715/-/DC1, for codes). If the Poisson method is in turn problematic because individual probabilities have to be estimated and those estimates become larger than 1 for some individuals, there may be no other solution than the doubling-of-cases method with robust standard error estimation, but this needs extra programming and statistical expertise. In line with other commentators,7^,10 we discourage the use of the Zhang and Yu method, despite its ease of application and its appealing conceptual simplicity.

Conclusion

In this paper we have shown the problems of using odds ratios as an approximation of risk ratios in cohort studies and RCTs. Researchers, reviewers and journal editors should be aware of potential misinterpretation of odds ratios, especially when the incidence of the outcome is large. The problem often arises when researchers use logistic regression to adjust for potential confounders. Misinterpretation of odds ratios should be avoided by calculating adjusted risk ratios. Journal editors and statistical reviewers can play an important role in encouraging researchers to present risk ratios instead of odds ratios in cohort studies and RCTs.

• Odds ratios, often used in cohort studies and randomized controlled trials (RCTs), are often interpreted as risk ratios but always overestimate the risk ratio.

• We evaluated alternatives for logistic regression to obtain adjusted risk ratios to determine which method performed best in estimating the correct risk ratio and confidence interval.

• The Mantel–Haenszel risk ratio method, log–binomial regression, Poisson regression with robust standard errors, and the doubling-of-cases method with robust standard errors gave correct risk ratios and confidence intervals.

• To avoid any misinterpretation of odds ratios, adjusted risk ratios should be calculated and presented in cohort studies and RCTs.