Skip to main content

Main menu

  • Home
  • Content
    • Current issue
    • Past issues
    • Early releases
    • Collections
    • Sections
    • Blog
    • Infographics & illustrations
    • Podcasts
    • COVID-19 Articles
  • Authors & Reviewers
    • Overview for authors
    • Submission guidelines
    • Submit a manuscript
    • Forms
    • Editorial process
    • Editorial policies
    • Peer review process
    • Publication fees
    • Reprint requests
    • Open access
    • Patient engagement
  • Members & Subscribers
    • Benefits for CMA Members
    • CPD Credits for Members
    • Subscribe to CMAJ Print
    • Subscription Prices
    • Obituary notices
  • Alerts
    • Email alerts
    • RSS
  • JAMC
    • À propos
    • Numéro en cours
    • Archives
    • Sections
    • Abonnement
    • Alertes
    • Trousse média 2023
    • Avis de décès
  • CMAJ JOURNALS
    • CMAJ Open
    • CJS
    • JAMC
    • JPN

User menu

Search

  • Advanced search
CMAJ
  • CMAJ JOURNALS
    • CMAJ Open
    • CJS
    • JAMC
    • JPN
CMAJ

Advanced Search

  • Home
  • Content
    • Current issue
    • Past issues
    • Early releases
    • Collections
    • Sections
    • Blog
    • Infographics & illustrations
    • Podcasts
    • COVID-19 Articles
  • Authors & Reviewers
    • Overview for authors
    • Submission guidelines
    • Submit a manuscript
    • Forms
    • Editorial process
    • Editorial policies
    • Peer review process
    • Publication fees
    • Reprint requests
    • Open access
    • Patient engagement
  • Members & Subscribers
    • Benefits for CMA Members
    • CPD Credits for Members
    • Subscribe to CMAJ Print
    • Subscription Prices
    • Obituary notices
  • Alerts
    • Email alerts
    • RSS
  • JAMC
    • À propos
    • Numéro en cours
    • Archives
    • Sections
    • Abonnement
    • Alertes
    • Trousse média 2023
    • Avis de décès
  • Visit CMAJ on Facebook
  • Follow CMAJ on Twitter
  • Follow CMAJ on Pinterest
  • Follow CMAJ on Youtube
  • Follow CMAJ on Instagram
Practice

If we're so different, why do we keep overlapping? When 1 plus 1 doesn't make 2

Rory Wolfe and James Hanley
CMAJ January 08, 2002 166 (1) 65-66;
Rory Wolfe
Dr. Wolfe is with the Department of Epidemiology and Preventive Medicine, Monash University, Melbourne, Victoria, Australia. Dr. Hanley is with the Department of Epidemiology and Biostatistics, McGill University, Montreal, Que.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
James Hanley
Dr. Wolfe is with the Department of Epidemiology and Preventive Medicine, Monash University, Melbourne, Victoria, Australia. Dr. Hanley is with the Department of Epidemiology and Biostatistics, McGill University, Montreal, Que.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Tables
  • Responses
  • Metrics
  • PDF
Loading

In the last decade, guidelines for the presentation of statistical results in medical journals have emphasized confidence intervals (CIs) as an adjunct to, or even a replacement for, statistical tests and p values. Because of the intimate links between the 2 concepts, authors now use statements like “the 95% CI overlaps 0” where they would formerly have stated “the difference is not statistically significant at the 5% level.” Although this interchangeability is technically correct in 1-sample situations, it does not carry over fully to comparisons involving 2 samples. A frequently encountered misconception is that if 2 independent 95% CIs overlap each other, as they do in Fig. 1, then a statistical test of the difference will not be statistically significant at the 5% level.

Figure
  • Download figure
  • Open in new tab
  • Download powerpoint

Fig. 1: Group means with confidence intervals that overlap.

Why is this not necessarily so? Consider the means in 2 independent groups, meanA and meanB, with for simplicity meanA being the smaller of the 2. The 95% CI for the mean in group A is approximately given by meanA plus or minus twice the standard error of the mean for that group, SEA, and correspondingly for group B. A mathematical check for whether these CIs overlap is given by adding the distance 2SEA (from meanA to the upper bound of the CI) to 2SEB and comparing this sum with the distance between the 2 means, that is, meanB minus meanA (Fig. 2). The CIs overlap when

Figure
  • Download figure
  • Open in new tab
  • Download powerpoint

Fig. 2: Confidence intervals and comparison of 2 group means (hypothetical clinical trial data: SEA = SEB = 1.8, means differ by 3 SE; assuming n >> 30 and independent samples, the 2-sided p value for testing the difference in means is approximately 0.036). SE = standard error of the mean.

[Equation 1]

Embedded Image

But overlapping confidence intervals do not demonstrate that group means are not statistically significantly different from each other. In a 2-sample t-test to compare 2 means, significance is attained at the 0.05 level if the t statistic exceeds the critical value of about 2, which occurs when the difference between the means exceeds twice its standard error, namely, if

[Equation 2]

Embedded Image

This standard error reflects the fact that the standard error of a difference involves summing the standard error of each estimate, but doing so by “adding in quadrature,” for example,

[Equation 3]

Embedded Image

Thus, to evaluate the overlap of 2 95% CIs and to determine whether at the same time the difference between the means is significant at the 0.05 level, the following rough rule can be used:

[Equation 4]

Embedded Image

If SEA and SEB are equal, the condition is as follows:

[Equation 5]

Embedded Image

When one SE is 25% larger than the other, the boundaries are 3.2 and 4.5 times the smaller SE. As the lower boundary remains close to 3, Moses1 was prompted to display group means with error bars that were 1.5 SE around the mean in order to have a “by eye” test of significance between the 2 group means while presenting the information in the 2 groups separately.

Footnotes

  • This article has been peer reviewed.

    Contributors: Both authors independently conceived of the material for this article. Both were involved in writing the article, and both have seen and approved the final version.

    Competing interests: None declared.

Reference

  1. 1.↵
    Moses LE. Graphical methods in statistical analysis. Annu Rev Public Health 1987;8:309-53.
    OpenUrlCrossRefPubMed
PreviousNext
Back to top

In this issue

CMAJ
Vol. 166, Issue 1
8 Jan 2002
  • Table of Contents
  • Index by author
  • Canadian Adverse Drug Reaction Newsletter (85-94)

Article tools

Respond to this article
Print
Download PDF
Article Alerts
To sign up for email alerts or to access your current email alerts, enter your email address below:
Email Article

Thank you for your interest in spreading the word on CMAJ.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
If we're so different, why do we keep overlapping? When 1 plus 1 doesn't make 2
(Your Name) has sent you a message from CMAJ
(Your Name) thought you would like to see the CMAJ web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
If we're so different, why do we keep overlapping? When 1 plus 1 doesn't make 2
Rory Wolfe, James Hanley
CMAJ Jan 2002, 166 (1) 65-66;

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
‍ Request Permissions
Share
If we're so different, why do we keep overlapping? When 1 plus 1 doesn't make 2
Rory Wolfe, James Hanley
CMAJ Jan 2002, 166 (1) 65-66;
Digg logo Reddit logo Twitter logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like

Jump to section

  • Article
    • Footnotes
    • Reference
  • Figures & Tables
  • Responses
  • Metrics
  • PDF

Related Articles

  • Highlights of this issue
  • PubMed
  • Google Scholar

Cited By...

  • The incidence of rugby-related catastrophic injuries (including cardiac events) in South Africa from 2008 to 2011: a cohort study
  • Google Scholar

More in this TOC Section

  • Pneumopericardium as a complication of pericardiocentesis
  • A blistering variant of phlegmasia cerulea dolens from underlying squamous cell lung cancer
  • Parechovirus infections in infants
Show more Practice

Similar Articles

Collections

  • Topics
    • Research methods & statistics

 

View Latest Classified Ads

Content

  • Current issue
  • Past issues
  • Collections
  • Sections
  • Blog
  • Podcasts
  • Alerts
  • RSS
  • Early releases

Information for

  • Advertisers
  • Authors
  • Reviewers
  • CMA Members
  • CPD credits
  • Media
  • Reprint requests
  • Subscribers

About

  • General Information
  • Journal staff
  • Editorial Board
  • Advisory Panels
  • Governance Council
  • Journal Oversight
  • Careers
  • Contact
  • Copyright and Permissions
  • Accessibiity
  • CMA Civility Standards
CMAJ Group

Copyright 2023, CMA Impact Inc. or its licensors. All rights reserved. ISSN 1488-2329 (e) 0820-3946 (p)

All editorial matter in CMAJ represents the opinions of the authors and not necessarily those of the Canadian Medical Association or its subsidiaries.

To receive any of these resources in an accessible format, please contact us at CMAJ Group, 500-1410 Blair Towers Place, Ottawa ON, K1J 9B9; p: 1-888-855-2555; e: cmajgroup@cmaj.ca

Powered by HighWire