Letters

Not-so-surprising findings

Sebastian Schneeweiss and Daniel H. Solomon
CMAJ July 18, 2006 175 (2) 171-172; DOI: https://doi.org/10.1503/cmaj.1060090

Investigators who publish their research in medical journals usually report effect measures in the form of ratios, such as relative risk or rate ratio. It is well known that the magnitude of ratio measures depends on the underlying risk for the study outcome:1 the higher the baseline risk in a population, the lower the risk ratio for a constant excess risk (Fig. 1). Randomized controlled trials (RCTs) often include only low-risk patients, selected through stringent exclusion criteria, wheras epidemiologic studies usually include typical patients with multiple comorbidities. These differences in baseline risk lead to higher ratio effect measures in RCTs than in nonrandomized studies, which would explain some of the observations reported by Panagiotis Papanikolaou and associates.2

Figure

Fig. 1: Illustration of the decrease in rate ratio (RR) with an increasing base-line outcome rate when the rate difference (RD) is constant (at 0.3 per 100 patient-years) but the baseline rate in unexposed patients differs (0.1 or 1 per 100 patient-years). The bar at left represents data from the VIGOR trial.3

Of relevance to this point is their scenario 15 on the risk of myocardial infarction in the context of therapy with rofecoxib or naproxen.2Table 1 shows the difference in risk structure and baseline event rates between the VIGOR trial3(which focused on gastrointestinal effects and excluded patients with major cardiovascular conditions) and the nonrandomized study by Ray and colleagues.4 These differences resulted in numerically higher rate ratio measures in the RCT (because of the lower baseline event rate), despite smaller rate differences.

In meta-analytic comparisons between RCTs and nonrandomized studies that fail to adjust for the respective baseline risks, the interpretation of ratio effect measures may be misleading. It has previously been demonstrated that careful epidemiologic studies that mimic the exclusion criteria of RCTs are likely to result in the same effect sizes as the RCTs.5 The strength of many nonrandomized studies is their assessment of harms of medical interventions in populations that are usually excluded from RCTs.

REFERENCES

  1. 1.
    Barratt A, Wyer PC, Hatala R, et al; Evidence-Based Medicine Teaching Tips Working Group. Tips for learners of evidence-based medicine: 1. Relative risk reduction, absolute risk reduction and number needed to treat. CMAJ 2004;171(4):353-8.
  2. 2.
    Papanikolaou PN, Christidi GD, Ioannidis JPA. Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies. CMAJ 2006;174(5):635-41.
  3. 3.
    Bombardier C, Laine L, Reicin A, et al; VIGOR Study Group. Comparison of upper gastrointestinal toxicity of rofecoxib and naproxen in patients with rheumatoid arthritis. N Engl J Med 2000;343:1520-8.
  4. 4.
    Ray WA, Stein CM, Daugherty JR, et al. COX-2 selective non-steroidal anti-inflammatory drugs and risk of serious coronary heart disease. Lancet 2002;360:1071-3.
  5. 5.
    Britton A, McKee M, Black N, et al. Choosing between randomised and non-randomised studies: a systematic review. Health Technol Assess 1998;2(13);i-iv, 1-124.