Skip to main content

Main menu

  • Home
  • Content
    • Current issue
    • Past issues
    • Early releases
    • Collections
    • Sections
    • Blog
    • Infographics & illustrations
    • Podcasts
    • COVID-19 articles
    • Obituary notices
  • Authors & Reviewers
    • Overview for authors
    • Submission guidelines
    • Submit a manuscript
    • Forms
    • Editorial process
    • Editorial policies
    • Peer review process
    • Publication fees
    • Reprint requests
    • Open access
    • Patient engagement
  • Physicians & Subscribers
    • Benefits for Canadian physicians
    • CPD Credits for CMA Members
    • Subscribe to CMAJ Print
    • Subscription prices
    • Obituary notices
  • Alerts
    • Email alerts
    • RSS
  • JAMC
    • À propos
    • Numéro en cours
    • Archives
    • Sections
    • Abonnement
    • Alertes
    • Trousse média 2023
    • Avis de décès
  • CMAJ JOURNALS
    • CMAJ Open
    • CJS
    • JAMC
    • JPN

User menu

Search

  • Advanced search
CMAJ
  • CMAJ JOURNALS
    • CMAJ Open
    • CJS
    • JAMC
    • JPN
CMAJ

Advanced Search

  • Home
  • Content
    • Current issue
    • Past issues
    • Early releases
    • Collections
    • Sections
    • Blog
    • Infographics & illustrations
    • Podcasts
    • COVID-19 articles
    • Obituary notices
  • Authors & Reviewers
    • Overview for authors
    • Submission guidelines
    • Submit a manuscript
    • Forms
    • Editorial process
    • Editorial policies
    • Peer review process
    • Publication fees
    • Reprint requests
    • Open access
    • Patient engagement
  • Physicians & Subscribers
    • Benefits for Canadian physicians
    • CPD Credits for CMA Members
    • Subscribe to CMAJ Print
    • Subscription prices
    • Obituary notices
  • Alerts
    • Email alerts
    • RSS
  • JAMC
    • À propos
    • Numéro en cours
    • Archives
    • Sections
    • Abonnement
    • Alertes
    • Trousse média 2023
    • Avis de décès
  • Visit CMAJ on Facebook
  • Follow CMAJ on Twitter
  • Follow CMAJ on Pinterest
  • Follow CMAJ on Youtube
  • Follow CMAJ on Instagram
Research

Assessment of potential bias in research grant peer review in Canada

Robyn Tamblyn, Nadyne Girard, Christina J. Qian and James Hanley
CMAJ April 23, 2018 190 (16) E489-E499; DOI: https://doi.org/10.1503/cmaj.170901
Robyn Tamblyn
Institute of Health Services and Policy Research (Tamblyn), Canadian Institutes of Health Research, Ottawa, Ont.; Departments of Medicine (Tamblyn) and Epidemiology, and Biostatistics (Girard, Qian, Hanley), Faculty of Medicine, McGill University, Montréal, Que.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nadyne Girard
Institute of Health Services and Policy Research (Tamblyn), Canadian Institutes of Health Research, Ottawa, Ont.; Departments of Medicine (Tamblyn) and Epidemiology, and Biostatistics (Girard, Qian, Hanley), Faculty of Medicine, McGill University, Montréal, Que.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Christina J. Qian
Institute of Health Services and Policy Research (Tamblyn), Canadian Institutes of Health Research, Ottawa, Ont.; Departments of Medicine (Tamblyn) and Epidemiology, and Biostatistics (Girard, Qian, Hanley), Faculty of Medicine, McGill University, Montréal, Que.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
James Hanley
Institute of Health Services and Policy Research (Tamblyn), Canadian Institutes of Health Research, Ottawa, Ont.; Departments of Medicine (Tamblyn) and Epidemiology, and Biostatistics (Girard, Qian, Hanley), Faculty of Medicine, McGill University, Montréal, Que.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Tables
  • Responses
  • Metrics
  • PDF
Loading

Abstract

BACKGROUND: Peer review is used to determine what research is funded and published, yet little is known about its effectiveness, and it is suspected that there may be biases. We investigated the variability of peer review and factors influencing ratings of grant applications.

METHODS: We evaluated all grant applications submitted to the Canadian Institutes of Health Research between 2012 and 2014. The contribution of application, principal applicant and reviewer characteristics to overall application score was assessed after adjusting for the applicant’s scientific productivity.

RESULTS: Among 11 624 applications, 66.2% of principal applicants were male and 64.1% were in a basic science domain. We found a significant nonlinear association between scientific productivity and final application score that differed by applicant gender and scientific domain, with higher scores associated with past funding success and h-index and lower scores associated with female applicants and those in the applied sciences. Significantly lower application scores were also associated with applicants who were older, evaluated by female reviewers only (v. male reviewers only, −0.05 points, 95% confidence interval [CI] −0.08 to −0.02) or reviewers in scientific domains different from the applicant’s (−0.07 points, 95% CI −0.11 to −0.03). Significantly higher application scores were also associated with reviewer agreement in application score (0.23 points, 95% CI 0.20 to 0.26), the existence of reviewer conflicts (0.09 points, 95% CI 0.07 to 0.11), larger budget requests (0.01 points per $100 000, 95% CI 0.007 to 0.02), and resubmissions (0.15 points, 95% CI 0.14 to 0.17). In addition, reviewers with high expertise were more likely than those with less expertise to provide higher scores to applicants with higher past success rates (0.18 points, 95% CI 0.08 to 0.28).

INTERPRETATION: There is evidence of bias in peer review of operating grants that is of sufficient magnitude to change application scores from fundable to nonfundable. This should be addressed by training and policy changes in research funding.

See related article at www.cmaj.ca/lookup/doi/10.1503/cmaj.180188

  • Accepted November 23, 2017.
View Full Text
PreviousNext
Back to top

In this issue

Canadian Medical Association Journal: 190 (16)
CMAJ
Vol. 190, Issue 16
23 Apr 2018
  • Table of Contents
  • Index by author

Article tools

Respond to this article
Print
Download PDF
Article Alerts
To sign up for email alerts or to access your current email alerts, enter your email address below:
Email Article

Thank you for your interest in spreading the word on CMAJ.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Assessment of potential bias in research grant peer review in Canada
(Your Name) has sent you a message from CMAJ
(Your Name) thought you would like to see the CMAJ web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
Assessment of potential bias in research grant peer review in Canada
Robyn Tamblyn, Nadyne Girard, Christina J. Qian, James Hanley
CMAJ Apr 2018, 190 (16) E489-E499; DOI: 10.1503/cmaj.170901

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
‍ Request Permissions
Share
Assessment of potential bias in research grant peer review in Canada
Robyn Tamblyn, Nadyne Girard, Christina J. Qian, James Hanley
CMAJ Apr 2018, 190 (16) E489-E499; DOI: 10.1503/cmaj.170901
Digg logo Reddit logo Twitter logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like

Jump to section

  • Article
    • Abstract
    • Methods
    • Results
    • Interpretation
    • Acknowledgements:
    • Footnotes
    • References
  • Figures & Tables
  • Responses
  • Metrics
  • PDF

Related Articles

  • The foundation and consequences of gender bias in grant peer review processes
  • PubMed
  • Google Scholar

Cited By...

  • Grant Review Feedback: Appropriateness and Usefulness
  • Characteristics of Heart Failure Trials Associated With Under-Representation of Women as Lead Authors
  • Gender and other potential biases in peer review: cross-sectional analysis of 38 250 external peer review reports
  • Proportion of female recipients of resident-selected awards across Canada from 2000 to 2018: a retrospective observational study
  • Mentoring for postdoctoral researchers in rheumatology: the Emerging EULAR Network (EMEUNET) post-doc mentoring programme
  • Dissecting disparity: improvements towards gender parity in leadership and on the podium within the Canadian Orthopaedic Association
  • Gender in Science, Technology, Engineering, and Mathematics: Issues, Causes, Solutions
  • Author-Reviewer Homophily in Peer Review
  • Does Gender Bias Still Affect Women in Science?
  • The foundation and consequences of gender bias in grant peer review processes
  • Google Scholar

More in this TOC Section

  • Statin initiation and risk of incident kidney disease in patients with diabetes
  • Acute care related to cannabis use during pregnancy after the legalization of nonmedical cannabis in Ontario
  • Safer opioid supply via a biometric dispensing machine: a qualitative study of barriers, facilitators and associated outcomes
Show more Research

Similar Articles

Collections

  • Topics
    • Academic medicine
    • Canadian government
    • Gender & health
    • Health policy
    • Research methods & statistics

 

View Latest Classified Ads

Content

  • Current issue
  • Past issues
  • Collections
  • Sections
  • Blog
  • Podcasts
  • Alerts
  • RSS
  • Early releases

Information for

  • Advertisers
  • Authors
  • Reviewers
  • CMA Members
  • CPD credits
  • Media
  • Reprint requests
  • Subscribers

About

  • General Information
  • Journal staff
  • Editorial Board
  • Advisory Panels
  • Governance Council
  • Journal Oversight
  • Careers
  • Contact
  • Copyright and Permissions
CMAJ Group

Copyright 2023, CMA Impact Inc. or its licensors. All rights reserved. ISSN 1488-2329 (e) 0820-3946 (p)

All editorial matter in CMAJ represents the opinions of the authors and not necessarily those of the Canadian Medical Association or its subsidiaries.

To receive any of these resources in an accessible format, please contact us at CMAJ Group, 500-1410 Blair Towers Place, Ottawa ON, K1J 9B9; p: 1-888-855-2555; e: cmajgroup@cmaj.ca

CMA Civility, Accessibility, Privacy

 

Powered by HighWire