Skip to main content

Main menu

  • Home
  • Content
    • Current issue
    • Past issues
    • Early releases
    • Collections
    • Sections
    • Blog
    • Infographics & illustrations
    • Podcasts
    • COVID-19 Articles
  • Authors & Reviewers
    • Overview for authors
    • Submission guidelines
    • Submit a manuscript
    • Forms
    • Editorial process
    • Editorial policies
    • Peer review process
    • Publication fees
    • Reprint requests
    • Open access
    • Patient engagement
  • Members & Subscribers
    • Benefits for CMA Members
    • CPD Credits for Members
    • Subscribe to CMAJ Print
    • Subscription Prices
  • Alerts
    • Email alerts
    • RSS
  • JAMC
    • À propos
    • Numéro en cours
    • Archives
    • Sections
    • Abonnement
    • Alertes
    • Trousse média 2023
  • CMAJ JOURNALS
    • CMAJ Open
    • CJS
    • JAMC
    • JPN

User menu

Search

  • Advanced search
CMAJ
  • CMAJ JOURNALS
    • CMAJ Open
    • CJS
    • JAMC
    • JPN
CMAJ

Advanced Search

  • Home
  • Content
    • Current issue
    • Past issues
    • Early releases
    • Collections
    • Sections
    • Blog
    • Infographics & illustrations
    • Podcasts
    • COVID-19 Articles
  • Authors & Reviewers
    • Overview for authors
    • Submission guidelines
    • Submit a manuscript
    • Forms
    • Editorial process
    • Editorial policies
    • Peer review process
    • Publication fees
    • Reprint requests
    • Open access
    • Patient engagement
  • Members & Subscribers
    • Benefits for CMA Members
    • CPD Credits for Members
    • Subscribe to CMAJ Print
    • Subscription Prices
  • Alerts
    • Email alerts
    • RSS
  • JAMC
    • À propos
    • Numéro en cours
    • Archives
    • Sections
    • Abonnement
    • Alertes
    • Trousse média 2023
  • Visit CMAJ on Facebook
  • Follow CMAJ on Twitter
  • Follow CMAJ on Pinterest
  • Follow CMAJ on Youtube
  • Follow CMAJ on Instagram
Review

Tips for learners of evidence-based medicine: 3. Measures of observer variability (kappa statistic)

Thomas McGinn, Peter C. Wyer, Thomas B. Newman, Sheri Keitz, Rosanne Leipzig, Gordon Guyatt for and ; The Evidence-Based Medicine Teaching Tips Working Group
CMAJ November 23, 2004 171 (11) 1369-1373; DOI: https://doi.org/10.1503/cmaj.1031981
Thomas McGinn
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Peter C. Wyer
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Thomas B. Newman
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Sheri Keitz
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Rosanne Leipzig
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Gordon Guyatt for
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Tables
  • Related Content
  • Responses
  • Metrics
  • PDF
Loading

Article Information

vol. 171 no. 11 1369-1373
DOI 
https://doi.org/10.1503/cmaj.1031981
PubMed 
15557592
History 
  • Published online November 22, 2004.
Copyright & Usage 
© 2004 Canadian Medical Association or its licensors

Author Information

  1. Thomas McGinn,
  2. Peter C. Wyer,
  3. Thomas B. Newman,
  4. Sheri Keitz,
  5. Rosanne Leipzig,
  6. Gordon Guyatt for,
  7. The Evidence-Based Medicine Teaching Tips Working Group
  1. From the Department of Medicine, Division of General Internal Medicine (McGinn), and the Department of Geriatrics (Leipzig), Mount Sinai Medical Center, New York, NY; the Columbia University College of Physicians and Surgeons, New York, NY (Wyer); the Departments of Epidemiology and Biostatistics and of Pediatrics, University of California, San Francisco, San Francisco, Calif. (Newman); Durham Veterans Affairs Medical Center and Duke University Medical Center, Durham, NC (Keitz); and the Departments of Medicine and of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ont. (Guyatt)Members of the Evidence-Based Medicine Teaching Tips Working Group: Peter C. Wyer (project director), College of Physicians and Surgeons, Columbia University, New York, NY; Deborah Cook, Gordon Guyatt (general editor), Ted Haines, Roman Jaeschke, McMaster University, Hamilton, Ont.; Rose Hatala (internal review coordinator), University of British Columbia, Vancouver, BC; Robert Hayward (editor, online version), Bruce Fisher, University of Alberta, Edmonton, Alta.; Sheri Keitz (field test coordinator), Durham Veterans Affairs Medical Center and Duke University Medical Center, Durham, NC; Alexandra Barratt, University of Sydney, Sydney, Australia; Pamela Charney, Albert Einstein College of Medicine, Bronx, NY; Antonio L. Dans, University of the Philippines College of Medicine, Manila, The Philippines; Barnet Eskin, Morristown Memorial Hospital, Morristown, NJ; Jennifer Kleinbart, Emory University School of Medicine, Atlanta, Ga.; Hui Lee, formerly Group Health Centre, Sault Ste. Marie, Ont. (deceased); Rosanne Leipzig, Thomas McGinn, Mount Sinai Medical Center, New York, NY; Victor M. Montori, Mayo Clinic College of Medicine, Rochester, Minn.; Virginia Moyer, University of Texas, Houston, Tex.; Thomas B. Newman, University of California, San Francisco, San Francisco, Calif.; Jim Nishikawa, University of Ottawa, Ottawa, Ont.; Kameshwar Prasad, Arabian Gulf University, Manama, Bahrain; W. Scott Richardson, Wright State University, Dayton, Ohio; Mark C. Wilson, University of Iowa, Iowa City, Iowa

Statistics from Altmetric.com

Usage statistics: November 2004 to February 2023

AbstractFullPdf
Nov 2004111171241
Dec 20045853669
Jan 20058543427
Feb 20052427289
Mar 200512508428
Apr 20054323200
May 20051356251
Jun 20051275194
Jul 20051347262
Aug 20051342311
Sep 20058282200
Oct 20053302254
Nov 20050364225
Dec 20050326156
Jan 20090224107
Feb 20090240100
Mar 20090318147
Apr 20090339143
May 20090253117
Jun 20090231132
Jul 2009024393
Aug 20090174101
Sep 2009019485
Oct 2009024986
Nov 20090308108
Dec 2009019865
Jan 2010023783
Feb 2010027499
Mar 20100250109
Apr 2010017870
May 2010019379
Jun 2010017575
Jul 2010021392
Aug 2010019697
Sep 2010018085
Oct 2010016593
Nov 20100230108
Dec 2010017469
Jan 20110221112
Feb 2011022797
Mar 20110199133
Apr 2011014981
May 2011011873
Jun 2011010085
Jul 2011010981
Aug 20110144114
Sep 2011016492
Oct 2011022399
Nov 20110245114
Dec 2011019095
Jan 20120267129
Feb 20120284156
Mar 20120358174
Apr 20120378122
May 20120367149
Jun 20120289130
Jul 20120270126
Aug 20120252109
Sep 20120256120
Oct 20120273155
Nov 2012029893
Dec 2012022472
Jan 20130242106
Feb 2013021859
Mar 2013028197
Apr 2013032488
May 2013021470
Jun 2013013363
Jul 2013010437
Aug 2013015677
Sep 2013014867
Oct 2013021279
Nov 2013017377
Dec 201309143
Jan 2014013569
Feb 2014010562
Mar 2014011462
Apr 201409034
May 2014013343
Jun 201408247
Jul 2014012341
Aug 201408236
Sep 201409740
Oct 2014013932
Nov 2014014050
Dec 2014012932
Jan 2015015470
Feb 201509431
Mar 201507925
Apr 201509451
May 201506843
Jun 201506521
Jul 201507524
Aug 2015010526
Sep 201509341
Oct 201508638
Nov 2015014150
Dec 2015010028
Jan 2016016758
Feb 2016010641
Mar 201608642
Apr 2016010341
May 201609137
Jun 201607232
Jul 201608326
Aug 201608625
Sep 2016013833
Oct 2016017327
Nov 2016019036
Dec 2016016424
Jan 2017019151
Feb 2017016719
Mar 2017018923
Apr 2017018926
May 2017014826
Jun 2017010217
Jul 201709028
Aug 2017012726
Sep 2017011116
Oct 2017012934
Nov 2017013423
Dec 2017014611
Jan 2018022724
Feb 2018010219
Mar 2018013539
Apr 2018011041
May 2018010614
Jun 2018013426
Jul 2018011619
Aug 2018012819
Sep 2018011627
Oct 2018018216
Nov 2018021120
Dec 201809312
Jan 2019010020
Feb 201906117
Mar 201905932
Apr 201908033
May 201906926
Jun 201904715
Jul 201906329
Aug 201905819
Sep 201906115
Oct 201906923
Nov 201907925
Dec 201905018
Jan 202006717
Feb 202008122
Mar 2020011227
Apr 202006227
May 202007927
Jun 202005716
Jul 202009017
Aug 202008238
Sep 2020014739
Oct 2020010038
Nov 202008038
Dec 202009035
Jan 202108238
Feb 202107320
Mar 202108034
Apr 2021010145
May 202108430
Jun 202109326
Jul 202105223
Aug 202104514
Sep 202109143
Oct 202109544
Nov 202108143
Dec 202109935
Jan 202205825
Feb 202208915
Mar 2022010725
Apr 202209239
May 202207730
Jun 202206026
Jul 202206525
Aug 202206624
Sep 202209625
Oct 2022011733
Nov 202207820
Dec 202205222
Jan 2023010329
Feb 2023043

Cited By...

  • 373 Citations
  • Google Scholar
PreviousNext
Back to top

In this issue

Canadian Medical Association Journal: 171 (11)
CMAJ
Vol. 171, Issue 11
23 Nov 2004
  • Table of Contents
  • Index by author

Article tools

Respond to this article
Print
Download PDF
Article Alerts
To sign up for email alerts or to access your current email alerts, enter your email address below:
Email Article

Thank you for your interest in spreading the word on CMAJ.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Tips for learners of evidence-based medicine: 3. Measures of observer variability (kappa statistic)
(Your Name) has sent you a message from CMAJ
(Your Name) thought you would like to see the CMAJ web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
Tips for learners of evidence-based medicine: 3. Measures of observer variability (kappa statistic)
Thomas McGinn, Peter C. Wyer, Thomas B. Newman, Sheri Keitz, Rosanne Leipzig, Gordon Guyatt for
CMAJ Nov 2004, 171 (11) 1369-1373; DOI: 10.1503/cmaj.1031981

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
‍ Request Permissions
Share
Tips for learners of evidence-based medicine: 3. Measures of observer variability (kappa statistic)
Thomas McGinn, Peter C. Wyer, Thomas B. Newman, Sheri Keitz, Rosanne Leipzig, Gordon Guyatt for
CMAJ Nov 2004, 171 (11) 1369-1373; DOI: 10.1503/cmaj.1031981
Digg logo Reddit logo Twitter logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like

Jump to section

  • Article
    • Clinician learners' objectives
    • Tip 1: Defining the importance of kappa
    • Tip 2: Calculating kappa
    • Tip 3: Calculating chance agreement
    • Conclusions
    • Footnotes
    • References
  • Figures & Tables
  • Related Content
  • Responses
  • Metrics
  • PDF

Related Articles

  • Highlights of this issue
  • Tips for learning and teaching evidence-based medicine: introduction to the series
  • Correction
  • Tips for learners of evidence-based medicine: 4. Assessing heterogeneity of primary studies in systematic reviews and whether to combine their results
  • Tips for learners of evidence-based medicine: 1. Relative risk reduction, absolute risk reduction and number needed to treat
  • Tips for learners of evidence-based medicine: 2. Measures of precision (confidence intervals)
  • Tips for learners of evidence-based medicine: 5. The effect of spectrum of disease on the performance of diagnostic tests
  • PubMed
  • Google Scholar

Cited By...

  • A systematic review of medical and clinical research landscapes and quality in Malaysia and Indonesia [REALQUAMI]: the review protocol
  • Changes in medication administration error rates associated with the introduction of electronic medication systems in hospitals: a multisite controlled before and after study
  • An Interrater Reliability Study of Pulmonary Function Assessment With a Portable Spirometer
  • Updated meta-analysis on prevention of venous thromboembolism in ambulatory cancer patients
  • Comparison of informal caregiver and named nurse assessment of symptoms in elderly patients dying in hospital using the palliative outcome scale
  • Paediatric advance care planning survey: a cross-sectional examination of congruence and discordance between adolescents with HIV/AIDS and their families
  • Most elite athletes return to competition following operative management of meniscal tears: a systematic review
  • The use of antibiotic prophylaxis in hip arthroscopy is under-reported and lacks evidence-based guidelines: a systematic review and survey
  • Systematic review and meta-analysis of the effect of SGLT-2 inhibitors on microvascular outcomes in patients with type 2 diabetes: a review protocol
  • Effects of advanced life support versus basic life support on the mortality rates of patients with trauma in prehospital settings: a study protocol for a systematic review and meta-analysis
  • Effectiveness of a 'Do not interrupt bundled intervention to reduce interruptions during medication administration: a cluster randomised controlled feasibility study
  • A systematic review and meta-analysis of trials of social network interventions in type 2 diabetes
  • Surgical treatment of femoroacetabular impingement following slipped capital femoral epiphysis: A systematic review
  • Answering medical questions at the point of care: a cross-sectional study comparing rapid decisions based on PubMed and Epistemonikos searches with evidence-based recommendations developed with the GRADE approach
  • Prospective validation of a clinical decision rule to identify patients presenting to the emergency department with chest pain who can safely be removed from cardiac monitoring
  • Athletic groin pain: a systematic review of surgical diagnoses, investigations and treatment
  • Safety and efficacy of tranexamic acid in bleeding paediatric trauma patients: a systematic review protocol
  • Return to Work and Sport Following High Tibial Osteotomy: A Systematic Review
  • Early evidence reports positive outcomes after osteochondral grafts and chondrocyte transplantation in the hip: a systematic review
  • Retractions in orthopaedic research: A systematic review
  • Use of heparins in patients with cancer: individual participant data meta-analysis of randomised trials study protocol
  • Childhood Cancer Mortality in India: Direct Estimates From a Nationally Representative Survey of Childhood Deaths
  • No difference in outcome between early versus delayed weight-bearing following microfracture surgery of the hip, knee or ankle: a systematic review of outcomes and complications
  • Expanding the scope of Critical Care Rapid Response Teams: a feasible approach to identify adverse events. A prospective observational cohort
  • Non-visible versus visible haematuria and bladder cancer risk: a study of electronic records in primary care
  • Computed tomography to assess risk of death in acute pulmonary embolism: a meta-analysis
  • The Correlation Between Esophageal and Abdominal Pressures in Mechanically Ventilated Patients Undergoing Laparoscopic Surgery
  • Determining the frequency and preventability of adverse drug reaction-related admissions to an Irish University Hospital: a cross-sectional study
  • Periprocedural Heparin Bridging in Patients Receiving Vitamin K Antagonists: Systematic Review and Meta-Analysis of Bleeding and Thromboembolic Rates
  • Authors response
  • Errors in the administration of intravenous medications in hospital and the role of correct procedures and nurse experience
  • Bedside Screening for Fistula Stenosis Should Be Tailored to the Site of the Arteriovenous Anastomosis
  • In Search of an Optimal Bedside Screening Program for Arteriovenous Fistula Stenosis
  • Homocysteine in Vascular Behcet Disease: A Meta-Analysis
  • Visual Assessment of the Spine Bruckel Instrument, a Novel Status Tool to Reflect Appearance of the Spine in Patients with Ankylosing Spondylitis
  • JAK2V617F mutation for the early diagnosis of Ph- myeloproliferative neoplasms in patients with venous thromboembolism: a meta-analysis
  • Relationship Between Preventability of Death After Coronary Artery Bypass Graft Surgery and All-Cause Risk-Adjusted Mortality Rates
  • Cardiovascular Risk Factors and Venous Thromboembolism: A Meta-Analysis
  • Accuracy of Physical Examination in the Detection of Arteriovenous Fistula Stenosis
  • Erythropoietin-receptor agonists in critically ill patients: a meta-analysis of randomized controlled trials
  • Discordance in Perceptions of Barriers to Diabetes Care Between Patients and Primary Care and Secondary Care
  • Thrombophilic abnormalities, oral contraceptives, and risk of cerebral vein thrombosis: a meta-analysis
  • Not Working 3 Years After Breast Cancer: Predictors in a Population-Based Study
  • Reliability of the Whiff Test in Clinical Practice
  • Teaching evidence-based practice on foot
  • Work absence after breast cancer diagnosis: a population-based study
  • Teaching evidence-based practice on foot
  • Kappa statistic
  • Kappa statistic
  • Kappa statistic
  • Kappa statistic
  • Editor's note
  • Google Scholar

More in this TOC Section

  • Diagnosis and acute management of migraine
  • Transient ischemic attack and minor stroke: diagnosis, risk stratification and management
  • Pre-exposure prophylaxis for HIV: effective and underused
Show more Review

Similar Articles

 

View Latest Classified Ads

Content

  • Current issue
  • Past issues
  • Collections
  • Sections
  • Blog
  • Podcasts
  • Alerts
  • RSS
  • Early releases

Information for

  • Advertisers
  • Authors
  • Reviewers
  • CMA Members
  • CPD credits
  • Media
  • Reprint requests
  • Subscribers

About

  • General Information
  • Journal staff
  • Editorial Board
  • Advisory Panels
  • Governance Council
  • Journal Oversight
  • Careers
  • Contact
  • Copyright and Permissions
  • Accessibiity
  • CMA Civility Standards
CMAJ Group

Copyright 2023, CMA Impact Inc. or its licensors. All rights reserved. ISSN 1488-2329 (e) 0820-3946 (p)

All editorial matter in CMAJ represents the opinions of the authors and not necessarily those of the Canadian Medical Association or its subsidiaries.

To receive any of these resources in an accessible format, please contact us at CMAJ Group, 500-1410 Blair Towers Place, Ottawa ON, K1J 9B9; p: 1-888-855-2555; e: cmajgroup@cmaj.ca

Powered by HighWire