Skip to main content

Main menu

  • Home
  • COVID-19
    • Articles & podcasts
    • Blog posts
    • Collection
    • News
  • Content
    • Current issue
    • Past issues
    • Early releases
    • Collections
    • Sections
    • Blog
    • Infographics & illustrations
    • Podcasts
    • Classified ads
  • Authors
    • Overview for authors
    • Submission guidelines
    • Submit a manuscript
    • Forms
    • Editorial process
    • Editorial policies
    • Peer review process
    • Publication fees
    • Reprint requests
  • CMA Members
    • Overview for members
    • Earn CPD Credits
    • Print copies of CMAJ
    • Career Ad Discount
  • Subscribers
    • General information
    • View prices
  • Alerts
    • Email alerts
    • RSS
  • JAMC
    • À propos
    • Numéro en cours
    • Archives
    • Sections
    • Abonnement
    • Alertes
  • CMAJ JOURNALS
    • CMAJ Open
    • CJS
    • JAMC
    • JPN

User menu

Search

  • Advanced search
CMAJ
  • CMAJ JOURNALS
    • CMAJ Open
    • CJS
    • JAMC
    • JPN
CMAJ

Advanced Search

  • Home
  • COVID-19
    • Articles & podcasts
    • Blog posts
    • Collection
    • News
  • Content
    • Current issue
    • Past issues
    • Early releases
    • Collections
    • Sections
    • Blog
    • Infographics & illustrations
    • Podcasts
    • Classified ads
  • Authors
    • Overview for authors
    • Submission guidelines
    • Submit a manuscript
    • Forms
    • Editorial process
    • Editorial policies
    • Peer review process
    • Publication fees
    • Reprint requests
  • CMA Members
    • Overview for members
    • Earn CPD Credits
    • Print copies of CMAJ
    • Career Ad Discount
  • Subscribers
    • General information
    • View prices
  • Alerts
    • Email alerts
    • RSS
  • JAMC
    • À propos
    • Numéro en cours
    • Archives
    • Sections
    • Abonnement
    • Alertes
  • Visit CMAJ on Facebook
  • Follow CMAJ on Twitter
  • Follow CMAJ on Pinterest
  • Follow CMAJ on Youtube
  • Follow CMAJ on Instagram
Research

Outcomes for patients with the same disease treated inside and outside of randomized trials: a systematic review and meta-analysis

Natasha Fernandes, Dianne Bryant, Lauren Griffith, Mohamed El-Rabbany, Nisha M. Fernandes, Crystal Kean, Jacquelyn Marsh, Siddhi Mathur, Rebecca Moyer, Clare J. Reade, John J. Riva, Lyndsay Somerville and Neera Bhatnagar
CMAJ November 04, 2014 186 (16) E596-E609; DOI: https://doi.org/10.1503/cmaj.131693
Natasha Fernandes
Faculty of Medicine (Natasha Fernandes, Mathur), University of Ottawa, Ottawa, Ont.; Faculty of Health Sciences (Bryant, Marsh, Moyer) and Schulich School of Medicine and Dentistry (Bryant), The University of Western Ontario, London, Ont.; Department of Clinical Epidemiology and Biostatistics (Bryant, Griffith), Department of Medicine (Nisha Fernandes), Health Sciences Library (Bhatnagar), Department of Family Medicine (Riva) and Division of Gynecologic Oncology (Reade), McMaster University, Hamilton, Ont.; Faculty of Dentistry (El-Rabbany), University of Toronto, Toronto, Ont.; School of Medical and Applied Sciences (Kean), Central Queensland University, Rockhampton, Australia; Department of Orthopaedic Surgery (Somerville), London Health Sciences Centre, London, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • For correspondence: natasha.fernandes31@gmail.com
Dianne Bryant
Faculty of Medicine (Natasha Fernandes, Mathur), University of Ottawa, Ottawa, Ont.; Faculty of Health Sciences (Bryant, Marsh, Moyer) and Schulich School of Medicine and Dentistry (Bryant), The University of Western Ontario, London, Ont.; Department of Clinical Epidemiology and Biostatistics (Bryant, Griffith), Department of Medicine (Nisha Fernandes), Health Sciences Library (Bhatnagar), Department of Family Medicine (Riva) and Division of Gynecologic Oncology (Reade), McMaster University, Hamilton, Ont.; Faculty of Dentistry (El-Rabbany), University of Toronto, Toronto, Ont.; School of Medical and Applied Sciences (Kean), Central Queensland University, Rockhampton, Australia; Department of Orthopaedic Surgery (Somerville), London Health Sciences Centre, London, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lauren Griffith
Faculty of Medicine (Natasha Fernandes, Mathur), University of Ottawa, Ottawa, Ont.; Faculty of Health Sciences (Bryant, Marsh, Moyer) and Schulich School of Medicine and Dentistry (Bryant), The University of Western Ontario, London, Ont.; Department of Clinical Epidemiology and Biostatistics (Bryant, Griffith), Department of Medicine (Nisha Fernandes), Health Sciences Library (Bhatnagar), Department of Family Medicine (Riva) and Division of Gynecologic Oncology (Reade), McMaster University, Hamilton, Ont.; Faculty of Dentistry (El-Rabbany), University of Toronto, Toronto, Ont.; School of Medical and Applied Sciences (Kean), Central Queensland University, Rockhampton, Australia; Department of Orthopaedic Surgery (Somerville), London Health Sciences Centre, London, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Mohamed El-Rabbany
Faculty of Medicine (Natasha Fernandes, Mathur), University of Ottawa, Ottawa, Ont.; Faculty of Health Sciences (Bryant, Marsh, Moyer) and Schulich School of Medicine and Dentistry (Bryant), The University of Western Ontario, London, Ont.; Department of Clinical Epidemiology and Biostatistics (Bryant, Griffith), Department of Medicine (Nisha Fernandes), Health Sciences Library (Bhatnagar), Department of Family Medicine (Riva) and Division of Gynecologic Oncology (Reade), McMaster University, Hamilton, Ont.; Faculty of Dentistry (El-Rabbany), University of Toronto, Toronto, Ont.; School of Medical and Applied Sciences (Kean), Central Queensland University, Rockhampton, Australia; Department of Orthopaedic Surgery (Somerville), London Health Sciences Centre, London, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Nisha M. Fernandes
Faculty of Medicine (Natasha Fernandes, Mathur), University of Ottawa, Ottawa, Ont.; Faculty of Health Sciences (Bryant, Marsh, Moyer) and Schulich School of Medicine and Dentistry (Bryant), The University of Western Ontario, London, Ont.; Department of Clinical Epidemiology and Biostatistics (Bryant, Griffith), Department of Medicine (Nisha Fernandes), Health Sciences Library (Bhatnagar), Department of Family Medicine (Riva) and Division of Gynecologic Oncology (Reade), McMaster University, Hamilton, Ont.; Faculty of Dentistry (El-Rabbany), University of Toronto, Toronto, Ont.; School of Medical and Applied Sciences (Kean), Central Queensland University, Rockhampton, Australia; Department of Orthopaedic Surgery (Somerville), London Health Sciences Centre, London, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Crystal Kean
Faculty of Medicine (Natasha Fernandes, Mathur), University of Ottawa, Ottawa, Ont.; Faculty of Health Sciences (Bryant, Marsh, Moyer) and Schulich School of Medicine and Dentistry (Bryant), The University of Western Ontario, London, Ont.; Department of Clinical Epidemiology and Biostatistics (Bryant, Griffith), Department of Medicine (Nisha Fernandes), Health Sciences Library (Bhatnagar), Department of Family Medicine (Riva) and Division of Gynecologic Oncology (Reade), McMaster University, Hamilton, Ont.; Faculty of Dentistry (El-Rabbany), University of Toronto, Toronto, Ont.; School of Medical and Applied Sciences (Kean), Central Queensland University, Rockhampton, Australia; Department of Orthopaedic Surgery (Somerville), London Health Sciences Centre, London, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Jacquelyn Marsh
Faculty of Medicine (Natasha Fernandes, Mathur), University of Ottawa, Ottawa, Ont.; Faculty of Health Sciences (Bryant, Marsh, Moyer) and Schulich School of Medicine and Dentistry (Bryant), The University of Western Ontario, London, Ont.; Department of Clinical Epidemiology and Biostatistics (Bryant, Griffith), Department of Medicine (Nisha Fernandes), Health Sciences Library (Bhatnagar), Department of Family Medicine (Riva) and Division of Gynecologic Oncology (Reade), McMaster University, Hamilton, Ont.; Faculty of Dentistry (El-Rabbany), University of Toronto, Toronto, Ont.; School of Medical and Applied Sciences (Kean), Central Queensland University, Rockhampton, Australia; Department of Orthopaedic Surgery (Somerville), London Health Sciences Centre, London, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Siddhi Mathur
Faculty of Medicine (Natasha Fernandes, Mathur), University of Ottawa, Ottawa, Ont.; Faculty of Health Sciences (Bryant, Marsh, Moyer) and Schulich School of Medicine and Dentistry (Bryant), The University of Western Ontario, London, Ont.; Department of Clinical Epidemiology and Biostatistics (Bryant, Griffith), Department of Medicine (Nisha Fernandes), Health Sciences Library (Bhatnagar), Department of Family Medicine (Riva) and Division of Gynecologic Oncology (Reade), McMaster University, Hamilton, Ont.; Faculty of Dentistry (El-Rabbany), University of Toronto, Toronto, Ont.; School of Medical and Applied Sciences (Kean), Central Queensland University, Rockhampton, Australia; Department of Orthopaedic Surgery (Somerville), London Health Sciences Centre, London, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Rebecca Moyer
Faculty of Medicine (Natasha Fernandes, Mathur), University of Ottawa, Ottawa, Ont.; Faculty of Health Sciences (Bryant, Marsh, Moyer) and Schulich School of Medicine and Dentistry (Bryant), The University of Western Ontario, London, Ont.; Department of Clinical Epidemiology and Biostatistics (Bryant, Griffith), Department of Medicine (Nisha Fernandes), Health Sciences Library (Bhatnagar), Department of Family Medicine (Riva) and Division of Gynecologic Oncology (Reade), McMaster University, Hamilton, Ont.; Faculty of Dentistry (El-Rabbany), University of Toronto, Toronto, Ont.; School of Medical and Applied Sciences (Kean), Central Queensland University, Rockhampton, Australia; Department of Orthopaedic Surgery (Somerville), London Health Sciences Centre, London, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Clare J. Reade
Faculty of Medicine (Natasha Fernandes, Mathur), University of Ottawa, Ottawa, Ont.; Faculty of Health Sciences (Bryant, Marsh, Moyer) and Schulich School of Medicine and Dentistry (Bryant), The University of Western Ontario, London, Ont.; Department of Clinical Epidemiology and Biostatistics (Bryant, Griffith), Department of Medicine (Nisha Fernandes), Health Sciences Library (Bhatnagar), Department of Family Medicine (Riva) and Division of Gynecologic Oncology (Reade), McMaster University, Hamilton, Ont.; Faculty of Dentistry (El-Rabbany), University of Toronto, Toronto, Ont.; School of Medical and Applied Sciences (Kean), Central Queensland University, Rockhampton, Australia; Department of Orthopaedic Surgery (Somerville), London Health Sciences Centre, London, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
John J. Riva
Faculty of Medicine (Natasha Fernandes, Mathur), University of Ottawa, Ottawa, Ont.; Faculty of Health Sciences (Bryant, Marsh, Moyer) and Schulich School of Medicine and Dentistry (Bryant), The University of Western Ontario, London, Ont.; Department of Clinical Epidemiology and Biostatistics (Bryant, Griffith), Department of Medicine (Nisha Fernandes), Health Sciences Library (Bhatnagar), Department of Family Medicine (Riva) and Division of Gynecologic Oncology (Reade), McMaster University, Hamilton, Ont.; Faculty of Dentistry (El-Rabbany), University of Toronto, Toronto, Ont.; School of Medical and Applied Sciences (Kean), Central Queensland University, Rockhampton, Australia; Department of Orthopaedic Surgery (Somerville), London Health Sciences Centre, London, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Lyndsay Somerville
Faculty of Medicine (Natasha Fernandes, Mathur), University of Ottawa, Ottawa, Ont.; Faculty of Health Sciences (Bryant, Marsh, Moyer) and Schulich School of Medicine and Dentistry (Bryant), The University of Western Ontario, London, Ont.; Department of Clinical Epidemiology and Biostatistics (Bryant, Griffith), Department of Medicine (Nisha Fernandes), Health Sciences Library (Bhatnagar), Department of Family Medicine (Riva) and Division of Gynecologic Oncology (Reade), McMaster University, Hamilton, Ont.; Faculty of Dentistry (El-Rabbany), University of Toronto, Toronto, Ont.; School of Medical and Applied Sciences (Kean), Central Queensland University, Rockhampton, Australia; Department of Orthopaedic Surgery (Somerville), London Health Sciences Centre, London, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
Neera Bhatnagar
Faculty of Medicine (Natasha Fernandes, Mathur), University of Ottawa, Ottawa, Ont.; Faculty of Health Sciences (Bryant, Marsh, Moyer) and Schulich School of Medicine and Dentistry (Bryant), The University of Western Ontario, London, Ont.; Department of Clinical Epidemiology and Biostatistics (Bryant, Griffith), Department of Medicine (Nisha Fernandes), Health Sciences Library (Bhatnagar), Department of Family Medicine (Riva) and Division of Gynecologic Oncology (Reade), McMaster University, Hamilton, Ont.; Faculty of Dentistry (El-Rabbany), University of Toronto, Toronto, Ont.; School of Medical and Applied Sciences (Kean), Central Queensland University, Rockhampton, Australia; Department of Orthopaedic Surgery (Somerville), London Health Sciences Centre, London, Ont.
  • Find this author on Google Scholar
  • Find this author on PubMed
  • Search for this author on this site
  • Article
  • Figures & Tables
  • Related Content
  • Responses
  • Metrics
  • PDF
Loading

Abstract

Background: It is unclear whether participation in a randomized controlled trial (RCT), irrespective of assigned treatment, is harmful or beneficial to participants. We compared outcomes for patients with the same diagnoses who did (“insiders”) and did not (“outsiders”) enter RCTs, without regard to the specific therapies received for their respective diagnoses.

Methods: By searching the MEDLINE (1966–2010), Embase (1980–2010), CENTRAL (1960–2010) and PsycINFO (1880–2010) databases, we identified 147 studies that reported the health outcomes of “insiders” and a group of parallel or consecutive “outsiders” within the same time period. We prepared a narrative review and, as appropriate, meta-analyses of patients’ outcomes.

Results: We found no clinically or statistically significant differences in outcomes between “insiders” and “outsiders” in the 23 studies in which the experimental intervention was ineffective (standard mean difference in continuous outcomes −0.03, 95% confidence interval [CI] −0.1 to 0.04) or in the 7 studies in which the experimental intervention was effective and was received by both “insiders” and “outsiders” (mean difference 0.04, 95% CI −0.04 to 0.13). However, in 9 studies in which an effective intervention was received only by “insiders,” the “outsiders” experienced significantly worse health outcomes (mean difference −0.36, 95% CI −0.61 to −0.12).

Interpretation: We found no evidence to support clinically important overall harm or benefit arising from participation in RCTs. This conclusion refutes earlier claims that trial participants are at increased risk of harm.

When people are asked to participate in a randomized controlled trial (RCT), it is natural for them to ask several questions in return. How safe are these treatments? How many extra visits and tests must I undergo? Will the researchers keep my family doctor informed about what’s going on? What outcomes are to be measured, and do they include ones that are of interest to me as a patient?

These multiple questions can be summarized as follows: Would I fare better being treated within the trial (as an “insider”) or in routine clinical care outside it (as an “outsider”)? Patients may ask this question in 1 of 2 ways. The first is highly specific: “Am I better off receiving this specific treatment as an insider or as an outsider?” Alternatively, they might ask a more general question: “Am I better off having my illness managed, regardless of the specific treatment I would receive, as an insider or as an outsider?” These questions are highly appropriate, and both deserve to be asked and answered,1,2 especially given that nonsystematic reviews have suggested a possible “inclusion benefit” from participating in trials.3

These 2 specific patient questions are analogous to those posed by researchers asking whether treatments do more good than harm when applied under “ideal” circumstances (in explanatory trials) or in the “real world” of routine health care (in pragmatic trials). Vist and colleagues answered the explanatory question when their earlier review4 found no advantage or disadvantage from receiving the same treatment inside or outside an RCT. Left unanswered, however, was the broader, more pragmatic question. In our experience, trial participants are often offered new, as-yet-untested treatments that would not be available to them outside the trial. This review looks at the dilemma faced by these patients, which needs to be addressed before general conclusions can be drawn about trial safety.

Methods

Data sources and searches

We searched the following databases: MEDLINE (1966 to November 2010), Embase (1980 to November 2010), Cochrane Central Register of Controlled Trials (CENTRAL; 1960 to last quarter of 2010) and PsycINFO (1880 to November 2010). The search strategy for each database is available upon request to the corresponding author. Studies were eligible for inclusion if they reported the same set of outcomes for “insiders” and “outsiders,” either simultaneously or within 2 months, where “insiders” were patients with a particular diagnosis who entered an RCT (whether treated with the intervention or a comparator) and “outsiders” were patients with the same diagnosis who did not enter the RCT. To validate our search, we compared our yield with the list of articles reviewed by Vist and colleagues.4

Study selection

Working in pairs, we reviewed the resulting titles and abstracts to screen for eligibility. Two reviewers independently screened the full text of eligible articles, with an independent third adjudicator resolving disagreements. Agreement was summarized with a weighted kappa coefficient.

Data extraction

Our primary outcome was mortality, and secondary outcomes included patient-reported or other clinically important outcomes. We calculated the relative risk (RR), unless count data were not reported, in which case we extracted the authors’ RR. We used adjusted RRs whenever they were reported.5 When RRs could not be calculated, we assumed that the reported odds ratios (ORs) approximated the RR for low event-rate outcomes.

For continuous outcomes, we extracted mean between-group differences and their standard deviations. We created rules for calculating missing outcomes using various statistical measures that were reported (Table 1).

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 1:

Assumptions and imputations used to calculate data if missing from published report

Prespecified causes of heterogeneity

We used the I2 statistic to measure the extent of heterogeneity between studies, where I2 values of 25%, 50% and 75% indicated low, medium and high heterogeneity, respectively.6 In addition, we constructed a priori hypotheses to potentially explain between-study heterogeneity, based on differences in types of outcomes, methodologic quality, types of care provided, potential for detection bias (due to differential follow-up or use of better diagnostic tools), potential for exclusion bias (if patients were excluded after enrolment because of characteristics related to outcome), potential for selection bias (due to imbalance of baseline characteristics), medical specialty and treatments provided.

In particular, we proposed 6 subgroups to explain observed heterogeneity due to treatment effect:

  1. when the randomized experimental intervention given to “insiders” was effective (i.e., the outcome was statistically significantly superior to the comparator), and “outsiders” received that same intervention or comparator

  2. when the randomized experimental intervention was effective, and “outsiders” received that same effective intervention only (without the comparator that was provided within the RCT)

  3. when the randomized experimental intervention was effective, and “outsiders” received the less effective comparator intervention only (without the experimental intervention provided within the RCT)

  4. when the randomized experimental intervention was effective, and “outsiders” received a different intervention (this subgroup acted as a positive control for the current analysis, since we anticipated better outcomes in the RCT group)

  5. when the randomized experimental and comparator interventions generated equivalent outcomes, with no further subdivision of this group (because any differences in outcomes between those treated inside and outside the RCT could be attributed to a trial effect)

  6. when insufficient information was provided about the effectiveness of the treatment in the trial and/or insufficient details were provided about the interventions received by “outsiders”

Data synthesis and analysis

Statistical calculations were performed with SPSS (version 20).7 Forest plots and funnel plots were created using Review Manager (version 5.1).8 When event counts were available, we used the Mantel–Haenszel method to estimate overall RR.9 If a study had a zero event rate in one group, we added a 0.5 correction to all cells. If only estimates of effect size and standard errors were provided, we used the generic inverse-variance meta-analysis function of Review Manager 5.1. We used the random-effects model to summarize outcomes.9

We first separated the studies into 2 groups according to whether randomization was applied in determining whether potential participants would be “insiders” or “outsiders.” Next, we separated studies by type of outcome: continuous or dichotomous, with the latter being further subdivided as nonmortality or mortality.

We created a funnel plot and conducted a sensitivity analysis to determine the stability of our conclusions.

Results

Summary of evidence

Following elimination of duplicate records and exclusions on the basis of initial screening and full-text review, 147 articles met our eligibility criteria and provided sufficient information to be included in our analysis (Figure 1).10–156 Details for the 576 articles excluded after full-text review, including reasons for exclusion, are available upon request. The eligibility of the remaining 74 articles was uncertain, and they were not included in the analysis.

Figure 1:
  • Download figure
  • Open in new tab
  • Download powerpoint
Figure 1:

PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) flow chart of studies identified and included in the analysis.10–156 The reasons for exclusions at screening and full-text review are available upon request to the corresponding author.

For full-text screening, the calculated average of the weighted kappa for eligibility was 0.68. There was 83% raw agreement between reviewers in the data-extraction phase for outcomes.

In 5 of the 147 eligible studies, patients were randomly assigned to become “insiders” and “outsiders.”38,41,86,87,141 In the remaining 142 studies, patients became part of the “outsiders” group for a variety of reasons. Table 2 presents the details about each included study.

View this table:
  • View inline
  • View popup
Table 2:

Characteristics of included studies

We analyzed a total of 48 continuous outcomes and 99 dichotomous outcomes; of the dichotomous outcomes, 74 were nonmortality outcomes, 4 were recurring outcomes (such as relapse rates), and 21 were mortality outcomes.

Risk of bias

Sources of risk of bias are detailed by individual study in Appendix 1 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.131693/-/DC1). In terms of detection bias, about two-thirds of the studies (n = 100) employed identical follow-up strategies for “insiders” and “outsiders.” In terms of exclusion bias affecting “insiders,” 67 studies had no exclusions, 1 study employed a deliberate but appropriate exclusion, and 74 studies inappropriately excluded “insiders” unequally between treatment groups; for the remaining 5 studies, the details were unclear. Forest plots based on subgroups created for each of these sources of bias did not change the results described below.

Replication of earlier studies

As a method of calibrating our search strategies and statistical methods, we carried out analyses of our dataset that were restricted to “insiders” and “outsiders” receiving identical treatments. These restricted analyses replicated the results of previous studies by Vist and colleagues4 and Gross and associates.157

Outcomes for studies with participants not randomized as “insiders” or “outsiders”

Our initial pooled analyses revealed a high degree of between-study heterogeneity (p < 0.001, I 2 = 84% for studies with dichotomous mortality outcomes; p < 0.001, I2 = 70% for studies with dichotomous nonmortality outcomes; p < 0.001, I 2 = 88% for studies with continuous outcomes). In total, mortality was determined for 53 714 “insiders” and 25 817 “outsiders” (see Table 3 and Appendix 2, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.131693/-/DC1). Dichotomous nonmortality outcomes were reported for 30 253 “insiders” and 30 000 “outsiders” (see Table 4 and Appendix 3, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.131693/-/DC1). We present the results of our nonrandomized continuous outcomes and randomized comparisons according to treatment effects, by presenting the subgrouping that left the least amount of remaining heterogeneity. All other forest plots are available upon request.

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 3:

Summary of meta-analyses for studies with mortality as an outcome, without randomization of potential participants as “insiders” v. “outsiders” (subgroups based on effectiveness of trial treatment)

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 4:

Summary of meta-analyses for studies with dichotomous nonmortality outcomes, without randomization of potential participants as “insiders” v. “outsiders” (subgroups based on effectiveness of trial treatment)

Results for clinically relevant subgroups

The results for continuous outcomes are summarized by subgroup in Table 5 (see also Appendix 4, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.131693/-/DC1).

View this table:
  • View inline
  • View popup
  • Download powerpoint
Table 5:

Summary of meta-analyses for studies with continuous outcomes, without randomization of potential participants as “insiders” v. “outsiders” (subgroups based on effectiveness of trial treatment)

There were 7 studies in which the randomized experimental intervention given to “insiders” (n = 6626) was effective, and “outsiders” (n = 2293) received that same intervention or the comparator. The heterogeneity was low to moderate (p = 0.2, I 2 = 37%), and the pooled result indicated neither significant harm nor significant benefit attributable to being an “insider” or an “outsider” (standardized mean difference 0.04, 95% confidence interval [CI] −0.04 to 0.13).

There were 3 studies in which the randomized experimental intervention (given to 1391 “insiders”) was effective, and the 5072 “outsiders” received only that same effective intervention. In this subgroup, there was a high degree of heterogeneity (p < 0.001, I2 = 95%).

There were 4 studies in which the randomized experimental intervention was effective, and “outsiders” received only the less effective comparator. In these studies, the 5794 “insiders” (those assigned to receive the active intervention or comparator) experienced a positive effect of the intervention, but the 9035 “outsiders” were offered only the ineffective comparator. In this subgroup, there was also a high degree of heterogeneity (p = 0.01, I2 = 74%).

There were 9 studies in which the randomized experimental intervention had a positive effect inside the RCT, but “outsiders” received a completely different intervention or comparator. For these studies, results could be pooled for the 649 “insiders” and 188 “outsiders” (standardized mean difference −0.36, 95% CI −0.61 to −0.12, p = 0.08, I 2 = 43%). In this subgroup, “insiders” fared statistically significantly better than “outsiders.”

The largest subgroup consisted of 23 studies in which the randomized experimental and comparator interventions generated equivalent outcomes. In this subgroup, the 5 940 “insiders” and 11 927 “outsiders” were given both treatments, only the control or only the experimental treatment, or completely different interventions. Heterogeneity among these studies was low to moderate (p = 0.10, I 2 = 29%). The pooled result revealed neither net harm nor net benefit for “insiders” compared with “outsiders” (standardized mean difference −0.03, 95% CI −0.1 to 0.04).

For the final subgroup of 2 studies, it was unclear whether there was a treatment effect or which interventions the “outsiders” received. We requested additional information from the study authors, but as of the date of publication, were still awaiting this clarification.

Outcomes for studies with participants randomized as “insiders” or “outsiders”

In 5 studies, potential participants were randomly assigned to become “insiders” or “outsiders.” One of these studies used a continuous outcome, with no reported difference between the 180 “insiders” and 97 “outsiders” (95% CI −0.22 to 0.27). The remaining 4 studies reported dichotomous nonmortality outcomes, with a moderate degree of heterogeneity (p = 0.06, I 2 = 60%). Their overall pooled effect indicated neither harm nor benefit when patients were treated inside or outside a trial (RR 0.94, 95% CI 0.56 to 1.57).

Additional analyses

Our investigation into publication bias showed a lack of smaller studies (both positive and negative) in our study. Because the included studies were symmetric around the pooled estimate, we are confident that our estimates are valid.

Our sensitivity analysis confirmed the robust nature of our imputations. Removing the studies with imputed outcomes had no significant effect on our results. Similarly, the results were not affected by clinical specialty.

Interpretation

Our study has confirmed the earlier findings of Vist and colleagues4 and Gross and associates,157 who reported that when trial participants (“insiders”) and nonparticipants (“outsiders”) receive the same treatments, they experience similar outcomes. As such, there is neither a “trial advantage” nor a “guinea pig disadvantage” of participating in an RCT. Furthermore, we have shown that even when “insiders” and “outsiders” are offered different interventions, there is no disadvantage to trial participation.

Our findings do not support the theory of “inclusion benefits,” “protocol effects” or “care effects” proposed by other authors.3,158 We found no differences in outcomes that could be attributed to health care workers providing additional care to “insiders,” the setting in which “insiders” were treated or the closer follow-up and attention that “insiders” receive. Had there been better care because physicians were following strict study protocol, a difference would have been detected between the groups for whom treatments were identical and would have been amplified within the subgroup of studies in which detection bias and expertise bias were most probable.

As expected, our subanalysis of “insiders” and “outsiders” who received the same treatments confirmed the results of the Vist and Gross reviews.4,157 However, we suggest that their insistence on identical interventions for patients inside and outside of an RCT answered only a narrow, explanatory question. For our review, we posed a more pragmatic question: Will patients fare better being treated within a trial (as “insiders”) or in routine clinical care outside it (as “outsiders”), regardless of the treatment received? In other words, will they be “sacrificial guinea pigs,” or, conversely, will they enjoy an “inclusion benefit”? Or will they fare the same inside the RCT or outside it? Our pragmatic study supports the last of these options, that patients will, in general, fare just as well regardless of whether they are “insiders” or “outsiders.”

Stiller159 reported a beneficial effect on mortality for “insiders.” However, that conclusion was based on simply counting the number of studies in which “insiders” had lower mortality than “outsiders,” ignoring the size of each study. As such, smaller studies (which are more prone to type II error) were weighted the same as much larger studies. Our random-effects meta-analysis took into account the size and weight of each study, and we found no such benefit from trial participation.

Limitations

Although 68% of the studies included here employed identical follow-up protocols for both “insiders” and “outsiders,” some studies did not explicitly state whether “outsiders” included all eligible patients or only those for whom data could be obtained. If “outsiders” are more likely to become lost to follow-up, in part because they have died or suffered other adverse events, true trial advantages might be missed.

Conclusion

We found no evidence to support either clinically important harm or clinically important benefit when patients’ illnesses were managed inside or outside an RCT. These results can inform discussions between clinicians and the patients to whom they are offering entry into peer-reviewed, ethically conducted RCTs. These results are also relevant to the policies, procedures and actions of institutions, ethics committees and granting agencies that permit and support the execution of RCTs.

Our findings and conclusions are only as good as the publication base of relevant RCTs, and we look forward to the day when the proposals of Vickers160 and Altman and Cates161 are fully realized, with all trials registered and reported and with raw trial data made readily available. When that day arrives, our study should be repeated to determine the validity of the conclusions reached here.

Acknowledgments

The authors would like to thank Dr. David Sackett for initiating this project. His insight and guidance throughout development of the manuscript were invaluable resources.

Footnotes

  • Competing interests: John Riva has received an NCMIC Foundation grant for work unrelated to the study reported here. He is also a board member of the Ontario Chiropractic Association. Lyndsay Somerville receives salary support (through her institution) from Smith & Nephew Canada. No other competing interests declared.

  • This article has been peer reviewed.

  • Contributors: Neera Bhatnagar designed and carried out the search. Natasha Fernandes, Dianne Bryant, Mohamed El-Rabbany, Nisha Fernandes, Crystal Kean, Jacquelyn Marsh, Siddhi Mathur, Rebecca Moyer, Clare Reade, John Riva and Lyndsay Somerville chose the included studies and extracted data. Natasha Fernandes analyzed the data with supervision from Lauren Griffith. Natasha Fernandes wrote the primary draft of the protocol and manuscript, and all other authors edited and further developed these components. All authors approved the final version. Dianne Bryant supervised this project. All authors agree to act as guarantors of this paper.

  • Funding: This study was supported by an internal grant from the University of Western Ontario to Dianne Bryant; no external funding was received. Natasha Fernandes was supported by McMaster University, the Canadian Institutes of Health Research Frederick Banting and Charles Best Canadian Graduate Scholarship and an Ontario Graduate Scholarship.

  • Data sharing: The dataset is available from the corresponding author.

References

  1. ↵
    1. Schwartz D,
    2. Lellouch J
    . Explanatory and pragmatic attitudes in therapeutic trials. J Chronic Dis 1967;20:637–48.
    OpenUrlCrossRefPubMed
  2. ↵
    1. Sackett DL,
    2. Gent M
    . Controversy in counting and attributing events in clinical trials. N Engl J Med 1979;301:1410–2.
    OpenUrlCrossRefPubMed
  3. ↵
    1. Lantos JD
    . The “inclusion benefit” in clinical trials. J Pediatr 1999; 134:130–1.
    OpenUrlCrossRefPubMed
  4. ↵
    1. Vist GE,
    2. Bryant D,
    3. Somerville L,
    4. et al
    . Outcomes of patients who participate in randomized controlled trials compared to similar patients receiving similar interventions who do not participate. Cochrane Database Syst Rev 2008;(3):MR000009.
  5. ↵
    1. DiCenso A
    . Systematic overviews of the prevention and predictors of adolescent pregnancy [dissertation]. Waterloo (ON): University of Waterloo; 1995.
  6. ↵
    1. Higgins JPT,
    2. Thompson SG,
    3. Deeks JJ,
    4. et al
    . Measuring inconsistency in meta-analyses. BMJ 2003;327:557–60.
    OpenUrlFREE Full Text
  7. ↵
    SPSS base 10.0 for Windows user’s guide. Chicago: SPSS Inc.; 1999.
  8. ↵
    Review Manager for Microsoft Word. Version 5.2. Copenhagen: Cochrane Collaboration, Nordic Cochrane Centre; 2012.
  9. ↵
    1. Higgins JPT,
    2. Green S
    , editors. Cochrane handbook for systematic reviews of interventions. Version 5.1.0. Oxford (UK): The Cochrane Collaboration; 2011 [updated March 2011]. Available: www.cochrane-handbook.org (accessed 2012 Aug. 21).
  10. ↵
    1. Akaza H,
    2. Hinotsu S,
    3. Aso Y,
    4. et al
    . Bacillus Calmette–Guerin treatment of existing papillary bladder cancer and carcinoma in situ of the bladder. Four-year results. The Bladder Cancer BCG Study Group. Cancer 1995;75:552–9.
    OpenUrlCrossRefPubMed
    1. Amar D,
    2. Roistacher N,
    3. Burt ME,
    4. et al
    . Effects of diltiazem versus digoxin on dysrhythmias and cardiac function after pneumonectomy. Ann Thorac Surg 1997;63:1374–81.
    OpenUrlCrossRefPubMed
    1. Andersson G,
    2. Lundstrom P,
    3. Strom L
    . Internet-based treatment of headache: Does telephone contact add anything? Headache 2003; 43:353–61.
    OpenUrlCrossRefPubMed
    1. Antman K,
    2. Amato D,
    3. Wood W,
    4. et al
    . Selection bias in clinical trials. J Clin Oncol 1985;3:1142–7.
    OpenUrlAbstract/FREE Full Text
    1. Ashok PW,
    2. Kidd A,
    3. Flett GM,
    4. et al
    . A randomised comparison of medical abortion and surgical vacuum aspiration at 10–13 weeks gestation. Hum Reprod 2002;17:92–8.
    OpenUrlAbstract/FREE Full Text
    1. Bain C,
    2. Cooper KG,
    3. Parkin DE
    . A partially randomised patient preference trial of microwave endometrial ablation using local anaesthesia and intravenous sedation or general anaesthesia: a pilot study. Gynaecol Endosc 2001;10:223–8.
    OpenUrlCrossRef
    1. Bakker A,
    2. Spinhoven P,
    3. van Balkom AJ,
    4. et al
    . Cognitive therapy by allocation versus cognitive therapy by preference in the treatment of panic disorder. Psychother Psychosom 2000;69:240–3.
    OpenUrlCrossRefPubMed
    1. Balmukhanov SB,
    2. Beisebaev AA,
    3. Aitkoolova ZI,
    4. et al
    . Intratumoral and parametrial infusion of metronidazole in the radiotherapy of uterine cervix cancer: preliminary report. Int J Radiat Oncol Biol Phys 1989;16:1061–3.
    OpenUrlCrossRefPubMed
    1. Bannister CF,
    2. Brosius KK,
    3. Sigl JC,
    4. et al
    . The effect of bispectral index monitoring on anesthetic use and recovery in children anesthetized with sevoflurane in nitrous oxide. Anesth Analg 2001;92:877–81.
    OpenUrlCrossRefPubMed
    1. Bedi N,
    2. Chilvers C,
    3. Churchill R,
    4. et al
    . Assessing effectiveness of treatment of depression in primary care: partially randomised preference trial. Br J Psychiatry 2000;177:312–8.
    OpenUrlAbstract/FREE Full Text
    1. Bell R,
    2. Palma S
    . Antenatal exercise and birthweight. Aust N Z J Obstet Gynaecol 2000;40:70–3.
    OpenUrlCrossRefPubMed
    1. Bhattacharya S,
    2. Cameron IM,
    3. Mollison J,
    4. et al
    . Admission–discharge policies for hysteroscopic surgery: a randomised comparison of day case with in-patient admission. Eur J Obstet Gynecol Reprod Biol 1998;76:81–4.
    OpenUrlCrossRefPubMed
    1. Biasoli I,
    2. Franchi-Rezgui P,
    3. Sibon D,
    4. et al
    . Analysis of factors influencing inclusion of 102 patients with stage III/IV Hodgkin’s lymphoma in a randomised trial for first-line chemotherapy. Ann Oncol 2008;19:1915–20.
    OpenUrlAbstract/FREE Full Text
    1. Biederman J,
    2. Herzog DB,
    3. Rivinus TM,
    4. et al
    . Amitriptyline in the treatment of anorexia nervosa: a double-blind, placebo-controlled study. J Clin Psychopharmacol 1985;5:10–6.
    OpenUrlCrossRefPubMed
    1. Bijker N,
    2. Peterse JL,
    3. Fentiman IS,
    4. et al
    . Effects of patient selection on the applicability of results from a randomized clinical trial (EORTC 10853) investigating breast-conserving therapy for DCIS. Br J Cancer 2002;87:615–20.
    OpenUrlCrossRefPubMed
    1. Blichert-Toft M,
    2. Brincker H,
    3. Andersen JA,
    4. et al
    . A Danish randomised trial comparing breast-preserving therapy with mastectomy in mammary carcinoma. Preliminary results. Acta Oncol 1988;27:671–7.
    OpenUrlCrossRefPubMed
    1. Blumenthal JA,
    2. Jiang W,
    3. Babyak MA,
    4. et al
    . Stress management and exercise training in cardiac patients with myocardial ischemia. Effects on prognosis and evaluation of mechanisms. Arch Intern Med 1997;157:2213–23.
    OpenUrlCrossRefPubMed
    1. Boesen EH,
    2. Boesen SH,
    3. Frederiksen K,
    4. et al
    . Survival after a psychoeducational intervention for patients with cutaneous malignant melanoma: a replication study. J Clin Oncol 2007;25: 5698–703.
    OpenUrlAbstract/FREE Full Text
    1. Boezaart AP,
    2. Berry RA,
    3. Laubscher JJ,
    4. et al
    . Evaluation of anxiolysis and pain associated with combined peri- and retrobulbar eye block for cataract surgery. J Clin Anesth 1998;10:204–10.
    OpenUrlCrossRefPubMed
    1. Brinkhaus B,
    2. Witt CM,
    3. Jena S,
    4. et al
    . Acupuncture in patients with allergic rhinitis: a pragmatic randomised trial. Ann Allergy Asthma Immunol 2008;101:535–43.
    OpenUrlCrossRefPubMed
    1. Caplan DB,
    2. Buchanan CN
    . Treatment of lower respiratory tract infections due to Pseudomonas aeruginosa in patients with cystic fibrosis. Rev Infect Dis 1984;6(Suppl 3):S705–10.
    OpenUrlCrossRefPubMed
  11. Coronary artery surgery study (CASS) (1984): a randomised trial of coronary artery bypass surgery. Comparability of entry characteristics and survival in randomised patients and nonrandomised patients meeting randomization criteria. J Am Coll Cardiol 1984;3:114–28.
    OpenUrlCrossRefPubMed
    1. Chauhan SP,
    2. Rutherford SE,
    3. Hess LW,
    4. et al
    . Prophylactic intrapartum amnioinfusion for patients with oligohydramnios. A prospective randomised study. J Reprod Med 1992;37:817–20.
    OpenUrlPubMed
    1. Chesebro JH,
    2. Fuster V,
    3. Elveback LR,
    4. et al
    . Trial of combined warfarin plus dipyridamole or aspirin therapy in prosthetic heart valve replacement: danger of aspirin compared with dipyridamole. Am J Cardiol 1983;51:1537–41.
    OpenUrlCrossRefPubMed
    1. Chilvers C,
    2. Dewey M,
    3. Fielding K,
    4. et al
    . Antidepressant drugs and generic counselling for treatment of major depression in primary care: randomized trial with patient preference arms. BMJ 2001;322:772–5.
    OpenUrlAbstract/FREE Full Text
    1. Clagett GP,
    2. Youkey JR,
    3. Brigham RA,
    4. et al
    . Asymptomatic cervical bruit and abnormal ocular pneumoplethysmography: a prospective study comparing two approaches to management. Surgery 1984;96:823–30.
    OpenUrlPubMed
    1. Clapp DW,
    2. Kliegman RM,
    3. Baley JE,
    4. et al
    . Use of intravenously administered immune globulin to prevent nosocomial sepsis in low birth weight infants: report of a pilot study. J Pediatr 1989;115:973–8.
    OpenUrlCrossRefPubMed
    1. Clemens JD,
    2. van Loon FF,
    3. Rao M,
    4. et al
    . Nonparticipation as a determinant of adverse health outcomes in a field trial of oral cholera vaccines. Am J Epidemiol 1992;135:865–74.
    OpenUrlAbstract/FREE Full Text
  12. ↵
    1. Cooper KG,
    2. Grant AM,
    3. Garratt AM
    . The impact of using a partially randomised patient preference design when evaluating alternative managements for heavy menstrual bleeding. Br J Obstet Gynaecol 1997;104:1367–73.
    OpenUrlCrossRefPubMed
    1. Cowchock FS,
    2. Reece EA,
    3. Balaban D,
    4. et al
    . Repeated fetal losses associated with antiphospholipid antibodies: a collaborative randomised trial comparing prednisone with low-dose heparin treatment. Am J Obstet Gynecol 1992;166:1318–23.
    OpenUrlCrossRefPubMed
    1. Creutzig U,
    2. Ritter J,
    3. Zimmermann M,
    4. et al
    . Does cranial irradiation reduce the risk for bone marrow relapse in acute myelogenous leukemia? Unexpected results of the childhood acute myelogenous leukemia study BFM-87. J Clin Oncol 1993;11: 279–86.
    OpenUrlAbstract/FREE Full Text
  13. ↵
    1. Dahan R,
    2. Caulin C,
    3. Figea L,
    4. et al
    . Does informed consent influence therapeutic outcome? A clinical trial of the hypnotic activity of placebo in patients admitted to hospital. BMJ Clin Res Ed 1986;293:363–4.
    OpenUrlAbstract/FREE Full Text
    1. Dalal HM,
    2. Evans PH,
    3. Campbell JL,
    4. et al
    . Home-based versus hospital-based rehabilitation after myocardial infarction: a randomised trial with preference arms — Cornwall Heart Attack Rehabilitation Management Study (CHARMS). Int J Cardiol 2007;119:202–11.
    OpenUrlCrossRefPubMed
    1. Decensi A,
    2. Robertson C,
    3. Viale G,
    4. et al
    . A randomised trial of low-dose tamoxifen on breast cancer proliferation and blood estrogenic biomarkers. J Natl Cancer Inst 2003;95:779–90.
    OpenUrlAbstract/FREE Full Text
    1. Detre KM,
    2. Guo P,
    3. Holubkov R,
    4. et al
    . Coronary revascularization in diabetic patients: a comparison of the randomised and observational components of the Bypass Angioplasty Revascularization Investigation (BARI). Circulation 1999;99:633–40.
    OpenUrlAbstract/FREE Full Text
    1. Loeffler M,
    2. Diehl V,
    3. Pfreundschuh M,
    4. et al
    . Dose–response relationship of complementary radiotherapy following four cycles of combination chemotherapy in intermediate-stage Hodgkin’s disease. J Clin Oncol 1997;15:2275–87.
    OpenUrlAbstract/FREE Full Text
    1. Eberhardt K,
    2. Rydgren L,
    3. Fex E,
    4. et al
    . d-Penicillamine in early rheumatoid arthritis: experience from a 2-year double blind placebo controlled study. Clin Exp Rheumatol 1996;14:625–31.
    OpenUrlPubMed
    1. Edsmyr F,
    2. Esposti PL,
    3. Johansson B,
    4. et al
    . Clinical experimental randomised study of 2,6-cis-diphenylhexamethylcyclotetrasiloxane and estramustine-17-phosphate in the treatment of prostatic carcinoma. J Urol 1978;120:705–7.
    OpenUrlPubMed
    1. Ekstein S,
    2. Elami A,
    3. Merin G,
    4. et al
    . Balloon angioplasty versus bypass grafting in the era of coronary stenting. Isr Med Assoc J 2002;4:583–9.
    OpenUrlPubMed
    1. Emery M,
    2. Beran MD,
    3. Darwiche J,
    4. et al
    . Results from a prospective, randomised, controlled study evaluating the acceptability and effects of routine pre-IVF counselling. Hum Reprod 2003;18:2647–53.
    OpenUrlAbstract/FREE Full Text
    1. Euler AR,
    2. Mitchell DK,
    3. Kline R,
    4. et al
    . Prebiotic effect of fructo-oligosaccharide supplemented term infant formula at two concentrations compared with unsupplemented formula and human milk. J Pediatr Gastroenterol Nutr 2005;40:157–64.
    OpenUrlCrossRefPubMed
    1. Feit F,
    2. Brooks MM,
    3. Sopko G,
    4. et al
    . Long-term clinical outcome in the Bypass Angioplasty Revascularization Investigation Registry: comparison with the randomised trial. BARI Investigators. Circulation 2000;101:2795–802.
    OpenUrlAbstract/FREE Full Text
    1. Forbes GM,
    2. Collins BJ
    . Nitrous oxide for colonoscopy: a randomised controlled study. Gastrointest Endosc 2000;51:271–7.
    OpenUrlCrossRefPubMed
    1. Franz MJ,
    2. Monk A,
    3. Barry B,
    4. et al
    . Effectiveness of medical nutrition therapy provided by dietitians in the management of non–insulin-dependent diabetes mellitus: a randomised, controlled clinical trial. J Am Diet Assoc 1995;95:1009–17.
    OpenUrlCrossRefPubMed
    1. Gall CA,
    2. Weller D,
    3. Esterman A,
    4. et al
    . Patient satisfaction and health-related quality of life after treatment for colon cancer. Dis Colon Rectum 2007;50:801–9.
    OpenUrlCrossRefPubMed
    1. Girón M,
    2. Fernandez-Yanez A,
    3. Mana-Alvarenga S,
    4. et al
    . Efficacy and effectiveness of individual family intervention on social and clinical functioning and family burden in severe schizophrenia: a 2-year randomised controlled study. Psychol Med 2010;40:73–84.
    OpenUrlCrossRefPubMed
    1. Goodkin DE,
    2. Plencner S,
    3. Palmer-Saxerud J,
    4. et al
    . Cyclophosphamide in chronic progressive multiple sclerosis. Maintenance vs nonmaintenance therapy. Arch Neurol 1987;44:823–7.
    OpenUrlCrossRefPubMed
    1. Gossop M,
    2. Johns A,
    3. Green L
    . Opiate withdrawal: inpatient versus outpatient programmes and preferred versus random assignment to treatment. BMJ 1986;293:103–4.
    OpenUrlAbstract/FREE Full Text
    1. Grant AM,
    2. Wileman SM,
    3. Ramsay CR,
    4. et al
    . Minimal access surgery compared with medical management for chronic gastro-oesophageal reflux disease: UK collaborative randomised trial. BMJ 2008;337:a2664.
    OpenUrlAbstract/FREE Full Text
    1. Gunn TR,
    2. Thompson JM,
    3. Jackson H,
    4. et al
    . Does early hospital discharge with home support of families with preterm infants affect breastfeeding success? A randomised trial. Acta Paediatr 2000;89:1358–63.
    OpenUrlCrossRefPubMed
    1. Helsing M,
    2. Bergman B,
    3. Thaning L,
    4. et al
    . Quality of life and survival in patients with advanced non-small cell lung cancer receiving supportive care plus chemotherapy with carboplatin and etoposide or supportive care only. A multicentre randomised phase III trial. Eur J Cancer 1998;34:1036–44.
    OpenUrlCrossRefPubMed
    1. Henriksson P,
    2. Edhaq O
    . Orchidectomy versus oestrogen for prostatic cancer: cardiovascular effects. BMJ 1986;293:413–5.
    OpenUrlAbstract/FREE Full Text
    1. Heuss LT,
    2. Drewe J,
    3. Schnieper P,
    4. et al
    . Patient-controlled versus nurse-administered sedation with propofol during colonoscopy. A prospective randomised trial. Am J Gastroenterol 2004; 99:511–8.
    OpenUrlCrossRefPubMed
    1. Hoh R,
    2. Pelfini A,
    3. Neese RA,
    4. et al
    . De novo lipogenesis predicts short-term body-composition response by bioelectrical impedance analysis to oral nutritional supplements in HIV-associated wasting. Am J Clin Nutr 1998;68:154–63.
    OpenUrlAbstract
    1. Howard LM,
    2. Flach C,
    3. Leese M,
    4. et al
    . Effectiveness and cost-effectiveness of admissions to women’s crisis houses compared with traditional psychiatric wards: pilot patient-preference randomised controlled trial. J Nerv Ment Dis 2009;197:722–7.
    OpenUrlCrossRefPubMed
    1. Howie FL,
    2. Henshaw RC,
    3. Naji SA,
    4. et al
    . Medical abortion or vacuum aspiration? Two year follow up of a patient preference trial. Br J Obstet Gynaecol 1997;104:829–33.
    OpenUrlCrossRefPubMed
    1. Jena S,
    2. Witt CM,
    3. Brinkhaus B,
    4. et al
    . Acupuncture in patients with headache. Cephalalgia 2008;28:969–79.
    OpenUrlAbstract/FREE Full Text
    1. Jensen LB,
    2. Vestergaard P,
    3. Hermann AP,
    4. et al
    . Hormone replacement therapy dissociates fat mass and bone mass, and tends to reduce weight gain in early postmenopausal women: a randomised controlled 5-year clinical trial of the Danish Osteoporosis Prevention Study. J Bone Miner Res 2003;18:333–42.
    OpenUrlCrossRefPubMed
    1. Kane WJ
    . Direct current electrical bone growth stimulation for spinal fusion. Spine 1988;13:363–5.
    OpenUrlCrossRefPubMed
    1. Karande VC,
    2. Korn A,
    3. Morris R,
    4. et al
    . Prospective randomised trial comparing the outcome and cost of in vitro fertilization with that of a traditional treatment algorithm as first-line therapy for couples with infertility. Fertil Steril 1999;71:468–75.
    OpenUrlCrossRefPubMed
    1. Kayser B,
    2. Hulsebosch R,
    3. Bosch F
    . Low-dose acetylsalicylic acid analog and acetazolamide for prevention of acute mountain sickness. High Alt Med Biol 2008;9:15–23.
    OpenUrlCrossRefPubMed
    1. Kendrick D,
    2. Fielding K,
    3. Bentley E,
    4. et al
    . The role of radiography in primary care patients with low back pain of at least 6 weeks duration: a randomised (unblinded) controlled trial. Health Technol Assess 2001;5:1–69.
    OpenUrlPubMed
    1. Kieler H,
    2. Hellberg D,
    3. Nilsson S,
    4. et al
    . Pregnancy outcome among non-participants in a trial on ultrasound screening. Ultrasound Obstet Gynecol 1998;11:104–9.
    OpenUrlCrossRefPubMed
    1. King M,
    2. Sibbald B,
    3. Ward E,
    4. et al
    . Randomised controlled trial of non-directive counselling, cognitive-behaviour therapy and usual general practitioner care in the management of depression as well as mixed anxiety and depression in primary care. Health Technol Assess 2000;4:1–83.
    OpenUrlPubMed
    1. Kirke PN,
    2. Daly LE,
    3. Elwood JH
    . A randomised trial of low dose folic acid to prevent neural tube defects. The Irish Vitamin Study Group. Arch Dis Child 1992;67:1442–6.
    OpenUrlAbstract/FREE Full Text
    1. Koch-Henriksen N,
    2. Sorensen PS,
    3. Christensen T,
    4. et al
    . A randomised study of two interferon-beta treatments in relapsing–remitting multiple sclerosis. Neurology 2006;66:1056–60.
    OpenUrlCrossRef
    1. Lansky D,
    2. Vance MA
    . School-based intervention for adolescent obesity: analysis of treatment, randomly selected control, and self-selected control subjects. J Consult Clin Psychol 1983;51:147–8.
    OpenUrlCrossRefPubMed
    1. Lichtenberg P,
    2. Levinson D,
    3. Sharshevsky Y,
    4. et al
    . Clinical case management of revolving door patients — a semi-randomised study. Acta Psychiatr Scand 2008;117:449–54.
    OpenUrlCrossRefPubMed
    1. Lidbrink E,
    2. Frisell J,
    3. Brandberg Y,
    4. et al
    . Nonattendance in the Stockholm mammography screening trial: relative mortality and reasons for nonattendance. Breast Cancer Res Treat 1995;35: 267–75.
    OpenUrlCrossRefPubMed
    1. Link MP,
    2. Goorin AM,
    3. Horowitz M,
    4. et al
    . Adjuvant chemotherapy of high-grade osteosarcoma of the extremity. Updated results of the Multi-Institutional Osteosarcoma Study. Breast Cancer Res Treat 1991;35:8–14.
    OpenUrl
    1. Liu PL,
    2. Shen WZ,
    3. Jing T,
    4. et al
    . Effects of compound Red-rooted Salvia and aspirin on platelet aggregation and PKB activity in the elderly patients with ACS. Chinese J New Drug 2009;18:900–2.
    OpenUrl
    1. Lock C,
    2. Wilson J,
    3. Steen N,
    4. et al
    . North of England and Scotland Study of Tonsillectomy and Adeno-tonsillectomy in Children (NESSTAC): a pragmatic randomised controlled trial with a parallel non-randomised preference study. Health Technol Assess 2010;14:1–164.
    OpenUrlPubMed
    1. Luby S,
    2. Agboatwalla M,
    3. Schnell BM,
    4. et al
    . The effect of antibacterial soap on impetigo incidence, Karachi, Pakistan. Am J Trop Med Hyg 2002;67:430–5.
    OpenUrlAbstract
    1. Macdonald JH,
    2. Marcora SM,
    3. Jibani MM,
    4. et al
    . Nandrolone decanoate as anabolic therapy in chronic kidney disease: a randomised phase II dose-finding study. Nephron Clin Pract 2007; 106:c125–35.
    OpenUrlCrossRefPubMed
    1. MacLennan AH,
    2. Kerin JF,
    3. Kirby C,
    4. et al
    . The effect of porcine relaxin vaginally applied at human embryo transfer in an in vitro fertilization programme. Aust N Z J Obstet Gynaecol 1985;25: 68–71.
    OpenUrlCrossRefPubMed
    1. MacMillan JF,
    2. Crow TJ,
    3. Johnson AL,
    4. et al
    . Short-term outcome in trial entrants and trial eligible patients. Br J Psychiatry 1986;148:128–33.
    OpenUrlAbstract/FREE Full Text
  14. ↵
    1. Mahon J,
    2. Laupacis A,
    3. Donner A,
    4. et al
    . Randomised study of n of 1 trials versus standard practice. BMJ 1996;312:1069–74.
    OpenUrlAbstract/FREE Full Text
  15. ↵
    1. Mahon JL,
    2. Laupacis A,
    3. Hodder RV,
    4. et al
    . Theophylline for irreversible chronic airflow limitation: a randomised study comparing n of 1 trials to standard practice. Chest 1999;115:38–48.
    OpenUrlCrossRefPubMed
    1. Marcinczyk MJ,
    2. Nicholas GG,
    3. Reed JF,
    4. et al
    . Asymptomatic carotid endarterectomy: patient and surgeon selection. Stroke 1997;28:291–6.
    OpenUrlAbstract/FREE Full Text
    1. Martin LF
    . Stress ulcers are common after aortic surgery. Endoscopic evaluation of prophylactic therapy. Am Surg 1994;60: 169–74.
    OpenUrlPubMed
    1. Martínez-Amenos A,
    2. Fernandez Ferre ML,
    3. Mota Vidal C,
    4. et al
    . Evaluation of two educative models in a primary care hypertension programme. J Hum Hypertens 1990;4:362–4.
    OpenUrlPubMed
    1. Masood J,
    2. Shah N,
    3. Lane T,
    4. et al
    . Nitrous oxide (Entonox) inhalation and tolerance of transrectal ultrasound guided prostate biopsy: a double-blind randomised controlled study. J Urol 2002;168:116–20.
    OpenUrlCrossRefPubMed
    1. Mattila PS,
    2. Joki-Erkkila VP,
    3. Kilpi T,
    4. et al
    . Prevention of otitis media by adenoidectomy in children younger than 2 years. Arch Otolaryngol Head Neck Surg 2003;129:163–8.
    OpenUrlCrossRefPubMed
  16. Mayo Asymptomatic Carotid Endarterectomy Study Group. Results of a randomised controlled trial of carotid endarterectomy for asymptomatic carotid stenosis. Mayo Clin Proc 1992;67:513–8.
    OpenUrlCrossRefPubMed
    1. McCaughey ES,
    2. Mulligan J,
    3. Voss LD,
    4. et al
    . Randomised trial of growth hormone in short normal girls. Lancet 1998;351: 940–4.
    OpenUrlCrossRefPubMed
    1. McKay JR,
    2. Alterman AI,
    3. McLellan AT,
    4. et al
    . Random versus nonrandom assignment in the evaluation of treatment for cocaine abusers. J Consult Clin Psychol 1998;66:697–701.
    OpenUrlCrossRefPubMed
    1. McKay JR,
    2. Alterman AI,
    3. McLellan AT,
    4. et al
    . Effect of random versus nonrandom assignment in a comparison of inpatient and day hospital rehabilitation for male alcoholics. J Consult Clin Psychol 1995;63:70–8.
    OpenUrlCrossRefPubMed
    1. Melchart D,
    2. Steger HG,
    3. Linde K,
    4. et al
    . Integrating patient preferences in clinical trials: a pilot study of acupuncture versus midazolam for gastroscopy. J Altern Complement Med 2002;8: 265–74.
    OpenUrlCrossRefPubMed
    1. Moertel CG,
    2. Childs DS,
    3. O’Fallon JR,
    4. et al
    . Combined 5-fluorouracil and radiation therapy as a surgical adjuvant for poor prognosis gastric carcinoma. J Clin Oncol 1984;2:1249–54.
    OpenUrlAbstract
    1. Mori A,
    2. Nobutoshi F,
    3. Asano T,
    4. et al
    . Cardiovascular tolerance in unsedated upper gastrointestinal endoscopy: prospective randomised comparison between transnasal and conventional oral procedures. Dig Endosc 2006;18:282–7.
    OpenUrlCrossRef
    1. Morrison DA,
    2. Sethi G,
    3. Sacks J,
    4. et al.
    VA AWESOME (Angina With Extremely Serious Operative Mortality Evaluation) Multicenter Registry. Percutaneous coronary intervention versus coronary bypass graft surgery for patients with medically refractory myocardial ischemia and risk factors for adverse outcomes with bypass: the VA AWESOME multi-center registry: comparison with the randomized clinical trial. J Am Coll Cardiol 2002;39:266–73.
    OpenUrlPubMed
    1. Nagel HT,
    2. Vandenbussche FP,
    3. Keirse MJ,
    4. et al
    . Amniocentesis before 14 completed weeks as an alternative to transabdominal chorionic villus sampling: a controlled trial with infant follow-up. Prenat Diagn 1998;18:465–75.
    OpenUrlCrossRefPubMed
    1. Neldam S,
    2. Osler M,
    3. Hansen PK,
    4. et al
    . Intrapartum fetal heart rate monitoring in a combined low- and high-risk population: a controlled clinical trial. Eur J Obstet Gynecol Reprod Biol 1986; 23:1–11.
    OpenUrlCrossRefPubMed
    1. Nicolaides K,
    2. Brizot ML,
    3. Patel F,
    4. et al
    . Comparison of chorionic villus sampling and amniocentesis for fetal karyotyping at 10–13 weeks’ gestation. Lancet 1994;344:435–9.
    OpenUrlCrossRefPubMed
    1. Ogden JA,
    2. Alvarez RG,
    3. Levitt RL,
    4. et al
    . Electrohydraulic high-energy shock-wave treatment for chronic plantar fasciitis. J Bone Joint Surg Am 2004;86A:2216–28.
    OpenUrlAbstract/FREE Full Text
    1. Palmon SC,
    2. Liu M,
    3. Moore LE,
    4. et al
    . Capnography facilitates tight control of ventilation during transport. Crit Care Med 1996; 24:608–11.
    OpenUrlCrossRefPubMed
    1. Panagopoulou E,
    2. Montgomery A,
    3. Tarlatzis B
    . Experimental emotional disclosure in women undergoing infertility treatment: Are drop outs better off? Soc Sci Med 2009;69:678–81.
    OpenUrlCrossRefPubMed
    1. Paradise JL,
    2. Bluestone CD,
    3. Bachman RZ,
    4. et al
    . Efficacy of tonsillectomy for recurrent throat infection in severely affected children. Results of parallel randomised and nonrandomised clinical trials. N Engl J Med 1984;310:674–83.
    OpenUrlCrossRefPubMed
    1. Petersen MK,
    2. Andersen KV,
    3. Andersen NT,
    4. et al
    . To whom do the results of this trial apply? External validity of a randomised controlled trial involving 130 patients scheduled for primary total hip replacement. Acta Orthop 2007;78:12–8.
    OpenUrlCrossRefPubMed
    1. Raistrick D,
    2. West D,
    3. Finnegan O,
    4. et al
    . A comparison of buprenorphine and lofexidine for community opiate detoxification: results from a randomised controlled trial. Addiction 2005;100:1860–7.
    OpenUrlCrossRefPubMed
    1. Reddihough DS,
    2. King J,
    3. Coleman G,
    4. et al
    . Efficacy of programmes based on conducive education for young children with cerebral palsy. Dev Med Child Neurol 1998;40:763–70.
    OpenUrlPubMed
    1. Rigg JR,
    2. Jamrozik K,
    3. Myles PS
    . Design of the multicenter Australian study of epidural anesthesia and analgesia in major surgery: the MASTER trial. Control Clin Trials 2000;21:244–56.
    OpenUrlCrossRefPubMed
    1. Rørbye C,
    2. Nørgaard M,
    3. Nilas L
    . Medical versus surgical abortion: comparing satisfaction and potential confounders in a partly randomised study. Hum Reprod 2005;20:834–8.
    OpenUrlAbstract/FREE Full Text
    1. Rosen MA,
    2. Roizen MF,
    3. Eger EI,
    4. et al
    . The effect of nitrous oxide on in vitro fertilization success rate. Anesthesiology 1987;67:42–4.
    OpenUrlCrossRefPubMed
    1. Salisbury C,
    2. Francis C,
    3. Rogers C,
    4. et al
    . A randomised controlled trial of clinics in secondary schools for adolescents with asthma. British J Gen Pract 2002;52:988–96.
    OpenUrl
    1. Sesso HD,
    2. Gaziano JM,
    3. VanDenburgh M,
    4. et al
    . Comparison of baseline characteristics and mortality experience of participants and nonparticipants in a randomised clinical trial: the Physicians’ Health Study. Control Clin Trials 2002;23:686–702.
    OpenUrlCrossRefPubMed
    1. Shain RN,
    2. Ratsula K,
    3. Toivonen J,
    4. et al
    . Acceptability of an experimental intracervical device: results of a study controlling for selection bias. Contraception 1989;39:73–84.
    OpenUrlCrossRefPubMed
    1. Smith P,
    2. Arnesen H
    . Mortality in non-consenters in a post-myocardial infarction trial. J Intern Med 1990;228:253–6.
    OpenUrlCrossRefPubMed
    1. Smuts CM,
    2. Borod E,
    3. Peeples JM,
    4. et al
    . High-DHA eggs: feasibility as a means to enhance circulating DHA in mother and infant. Lipids 2003;38:407–14.
    OpenUrlCrossRefPubMed
    1. Stecksén-Blicks C,
    2. Holgerson PL,
    3. Twetman S
    . Effect of xylitol and xylitol-fluoride lozenges on approximal caries development in high-caries-risk children. Int J Paediatr Dent 2008;18:170–7.
    OpenUrlCrossRefPubMed
    1. Stern C,
    2. Chamley L,
    3. Norris H,
    4. et al
    . A randomised, double-blind, placebo-controlled trial of heparin and aspirin for women with in vitro fertilization implantation failure and antiphospholipid or antinuclear antibodies. Fertil Steril 2003;80:376–83.
    OpenUrlCrossRefPubMed
    1. Stith SM,
    2. Rosen KH,
    3. McCollum EE,
    4. et al
    . Treating intimate partner violence within intact couple relationships: outcomes of multi-couple versus individual couple therapy. J Marital Fam Ther 2004;30:305–18.
    OpenUrlCrossRefPubMed
    1. Stockton KA,
    2. Mengersen KA
    . Effect of multiple physiotherapy sessions on functional outcomes in the initial postoperative period after primary total hip replacement: a randomised controlled trial. Arch Phys Med Rehabil 2009;90:1652–7.
    OpenUrlCrossRefPubMed
    1. Strandberg TE,
    2. Salomaa VV,
    3. Vanhanen HT,
    4. et al
    . Mortality in participants and non-participants of a multifactorial prevention study of cardiovascular diseases: a 28 year follow up of the Helsinki Businessmen Study. Br Heart J 1995;74:449–54.
    OpenUrlAbstract/FREE Full Text
    1. Suherman SK,
    2. Affandi B,
    3. Korver T
    . The effects of Implanon on lipid metabolism in comparison with Norplant. Contraception 1999;60:281–7.
    OpenUrlCrossRefPubMed
    1. Sullivan MP,
    2. Fuller LM,
    3. Chen T,
    4. et al
    . Intergroup Hodgkin’s disease in children study of stages I and II: a preliminary report. Cancer Treat Rep 1982;66:937–47.
    OpenUrlPubMed
    1. Sundar S,
    2. Rai M,
    3. Chakravarty J,
    4. et al
    . New treatment approach in Indian visceral leishmaniasis: single-dose liposomal amphotericin B followed by short-course oral miltefosine. Clin Infect Dis 2008;47:1000–6.
    OpenUrlAbstract/FREE Full Text
    1. Taddio A,
    2. Lee C,
    3. Yip A,
    4. et al
    . Intravenous morphine and topical tetracaine for treatment of pain in neonates undergoing central line placement. JAMA 2006;295:793–800.
    OpenUrlCrossRefPubMed
    1. Tanai C,
    2. Nokihara H,
    3. Yamamoto S,
    4. et al
    . Characteristics and outcomes of patients with advanced non–small-cell lung cancer who declined to participate in randomised clinical chemotherapy trials. Br J Cancer 2009;100:1037–42.
    OpenUrlCrossRefPubMed
    1. Tanaka S,
    2. Tsuchida H,
    3. Namba H,
    4. et al
    . Clonidine and lidocaine inhibition of isoflurane-induced tachycardia in humans. Anesthesiology 1994;81:1341–9.
    OpenUrlCrossRefPubMed
    1. Taplin D,
    2. Meinking TL,
    3. Castillero PM,
    4. et al
    . Permethrin 1% creme rinse for the treatment of Pediculus humanus var capitis infestation. Pediatr Dermatol 1986;3:344–8.
    OpenUrlCrossRefPubMed
    1. Tenenbaum A,
    2. Motro M,
    3. Fisman EZ,
    4. et al
    . Does participation in a long-term clinical trial lead to survival gain for patients with coronary artery disease? Am J Med 2002;112:545–8.
    OpenUrlCrossRefPubMed
    1. Toprak A,
    2. Erenus M,
    3. Ilhan AH,
    4. et al
    . The effect of postmenopausal hormone therapy with or without folic acid supplementation on serum homocysteine level. Climacteric 2005;8:279–86.
    OpenUrlCrossRefPubMed
    1. Underwood M,
    2. Ashby D,
    3. Carnes D,
    4. et al
    . Topical or oral ibuprofen for chronic knee pain in older people. The TOIB study. Health Technol Assess 2008;12: iii–iv, ix–155.
    OpenUrl
    1. Urban P,
    2. Stauffer JC,
    3. Bleed D,
    4. et al
    . A randomised evaluation of early revascularization to treat shock complicating acute myocardial infarction. The (Swiss) Multicenter Trial of Angioplasty for Shock-(S)MASH. Eur Heart J 1999;20:1030–8.
    OpenUrlAbstract/FREE Full Text
    1. van Bergen PF,
    2. Jonker JJ,
    3. Molhoek GP,
    4. et al
    . Characteristics and prognosis of non-participants of a multi-centre trial of long-term anticoagulant treatment after myocardial infarction. Int J Cardiol 1995;49:135–41.
    OpenUrlCrossRefPubMed
    1. Van HL,
    2. Dekker J,
    3. Koelen J,
    4. et al
    . Patient preference compared with random allocation in short-term psychodynamic supportive psychotherapy with indicated addition of pharmacotherapy for depression. Psychother Res 2009;19:205–12.
    OpenUrlCrossRefPubMed
    1. Verdonck LF,
    2. van Putten WL,
    3. Hagenbeek A,
    4. et al
    . Comparison of CHOP chemotherapy with autologous bone marrow transplantation for slowly responding patients with aggressive non-Hodgkin’s lymphoma. N Engl J Med 1995;332:1045–51.
    OpenUrlCrossRefPubMed
    1. Vind AB,
    2. Andersen HE,
    3. Pedersen KD,
    4. et al
    . Baseline and follow-up characteristics of participants and nonparticipants in a randomised clinical trial of multifactorial fall prevention in Denmark. J Am Geriatr Soc 2009;57:1844–9.
    OpenUrlCrossRefPubMed
    1. Walker WS,
    2. Raychaudhury T,
    3. Faichney A,
    4. et al
    . Wound colonisation following cardiac surgery. J Cardiovasc Surg (Torino) 1986;27:662–6.
    OpenUrlPubMed
    1. Wallage S,
    2. Cooper KG,
    3. Graham WJ,
    4. et al
    . A randomised trial comparing local versus general anaesthesia for microwave endometrial ablation. Int J Obstet Gynaecol 2003;110:799–807.
    OpenUrlCrossRef
  17. ↵
    1. Watzke B,
    2. Ruddel H,
    3. Jurgensen R,
    4. et al
    . Effectiveness of systematic treatment selection for psychodynamic and cognitive–behavioural therapy: randomised controlled trial in routine mental healthcare. Br J Psychiatry 2010;197:96–105.
    OpenUrlAbstract/FREE Full Text
    1. Welt SI,
    2. Dorminy JH,
    3. Jelovsek FR,
    4. et al
    . The effect of prophylactic management and therapeutics on hypertensive disease in pregnancy: preliminary studies. Obstet Gynecol 1981;57:557–65.
    OpenUrlPubMed
    1. West J,
    2. Wright J,
    3. Tuffnell D,
    4. et al
    . Do clinical trials improve quality of care? A comparison of clinical processes and outcomes in patients in a clinical trial and similar patients outside a trial where both groups are managed according to a strict protocol. Qual Saf Health Care 2005;14:175–8.
    OpenUrlAbstract/FREE Full Text
    1. Wetzner SM,
    2. Vincent ME,
    3. Robbins AH
    . Ceruletide-assisted cholecystography: a clinical assessment. Radiology 1979;131:23–6.
    OpenUrlCrossRefPubMed
  18. World Health Organization (WHO) Task Force on Oral Contraceptives. Effects of hormonal contraceptives on breast milk composition and infant growth. Stud Fam Plann 1988;19:361–9.
    OpenUrlCrossRefPubMed
    1. Wieringa-de Waard M,
    2. Vos J,
    3. Bensel GJ,
    4. et al
    . Management of miscarriage: a randomised controlled trial of expectant management versus surgical evacuation. Hum Reprod 2002;17:2445–50.
    OpenUrlAbstract/FREE Full Text
    1. Williford WO,
    2. Krol WF,
    3. Buzby GP
    . Comparison of eligible randomised patients with two groups of ineligible patients: Can the results of the VA Total Parenteral Nutrition clinical trial be generalized? J Clin Epidemiol 1993;46:1025–34.
    OpenUrlCrossRefPubMed
    1. Witt CM,
    2. Jena S,
    3. Brinkhaus B,
    4. et al
    . Acupuncture in patients with osteoarthritis of the knee or hip: a randomised, controlled trial with an additional nonrandomised arm. Arthritis Rheum 2006;54:3485–93.
    OpenUrlCrossRefPubMed
    1. Witt CM,
    2. Jena S,
    3. Brinkhaus B,
    4. et al
    . Acupuncture for patients with chronic neck pain. Pain 2006;125:98–106.
    OpenUrlCrossRefPubMed
    1. Witt CM,
    2. Jena S,
    3. Selim D,
    4. et al
    . Pragmatic randomised trial evaluating the clinical and economic effectiveness of acupuncture for chronic low back pain. Am J Epidemiol 2006;164:487–96.
    OpenUrlAbstract/FREE Full Text
    1. Witt CM,
    2. Reinhold T,
    3. Brinkhaus B,
    4. et al
    . Acupuncture in patients with dysmenorrhea: a randomised study on clinical effectiveness and cost-effectiveness in usual care. Am J Obstet Gynecol 2008;198:166 e1–8.
    OpenUrl
    1. Woodhouse SP,
    2. Cox S,
    3. Boyd P,
    4. et al
    . High dose and standard dose adrenaline do not alter survival, compared with placebo, in cardiac arrest. Resuscitation 1995;30:243–9.
    OpenUrlCrossRefPubMed
    1. Wyse DG,
    2. Hallstrom A,
    3. McBride R,
    4. et al
    . Events in the Cardiac Arrhythmia Suppression Trial (CAST): mortality in patients surviving open label titration but not randomised to double-blind therapy. J Am Coll Cardiol 1991;18:20–8.
    OpenUrlCrossRefPubMed
    1. Yamamoto H,
    2. Hughes RW,
    3. Schroeder KW,
    4. et al
    . Treatment of benign esophageal stricture by Eder-Puestow or balloon dilators: a comparison between randomised and prospective non-randomised trials. Mayo Clin Proc 1992;67:228–36.
    OpenUrlCrossRefPubMed
    1. Yamani MH,
    2. Avery R,
    3. Mawhorter SD,
    4. et al
    . The impact of CytoGam on cardiac transplant recipients with moderate hypogammaglobulinemia: a randomised single-center study. J Heart Lung Transplant 2005;24:1766–9.
    OpenUrlCrossRefPubMed
  19. ↵
    1. Yersin B,
    2. Besson J,
    3. Duc-Mingot S,
    4. et al
    . Screening and referral of alcoholic patients in a general hospital. Eur Addict Res 1996; 2:94–101.
    OpenUrlCrossRef
  20. ↵
    1. Gross CP,
    2. Krumholz HM,
    3. Van Wye G,
    4. et al
    . Does random treatment assignment cause harm to research participants? PLoS Med 2006;3:e188.
    OpenUrlCrossRefPubMed
  21. ↵
    1. Braunholtz DA,
    2. Edwards SJ,
    3. Lilford RJ,
    4. et al
    . Are randomized clinical trials good for us (in the short term)? Evidence for a “trial effect.” J Clin Epidemiol 2001;54:217–24.
    OpenUrlCrossRefPubMed
  22. ↵
    1. Stiller CA
    . Centralised treatment, entry to trials and survival. Br J Cancer 1994;70:352–62.
    OpenUrlCrossRefPubMed
  23. ↵
    1. Vickers AJ
    . Whose data set is it anyway? Sharing raw data from randomised trials. Trials 2006;7:15.
    OpenUrlCrossRefPubMed
  24. ↵
    1. Altman DG,
    2. Cates C
    . Authors should make their data available. BMJ 2001;323:1069–70.
    OpenUrlFREE Full Text
View Abstract
PreviousNext
Back to top

In this issue

Canadian Medical Association Journal: 186 (16)
CMAJ
Vol. 186, Issue 16
4 Nov 2014
  • Table of Contents
  • Index by author

Article tools

Respond to this article
Print
Download PDF
Article Alerts
To sign up for email alerts or to access your current email alerts, enter your email address below:
Email Article

Thank you for your interest in spreading the word on CMAJ.

NOTE: We only request your email address so that the person you are recommending the page to knows that you wanted them to see it, and that it is not junk mail. We do not capture any email address.

Enter multiple addresses on separate lines or separate them with commas.
Outcomes for patients with the same disease treated inside and outside of randomized trials: a systematic review and meta-analysis
(Your Name) has sent you a message from CMAJ
(Your Name) thought you would like to see the CMAJ web site.
CAPTCHA
This question is for testing whether or not you are a human visitor and to prevent automated spam submissions.
Citation Tools
Outcomes for patients with the same disease treated inside and outside of randomized trials: a systematic review and meta-analysis
Natasha Fernandes, Dianne Bryant, Lauren Griffith, Mohamed El-Rabbany, Nisha M. Fernandes, Crystal Kean, Jacquelyn Marsh, Siddhi Mathur, Rebecca Moyer, Clare J. Reade, John J. Riva, Lyndsay Somerville, Neera Bhatnagar
CMAJ Nov 2014, 186 (16) E596-E609; DOI: 10.1503/cmaj.131693

Citation Manager Formats

  • BibTeX
  • Bookends
  • EasyBib
  • EndNote (tagged)
  • EndNote 8 (xml)
  • Medlars
  • Mendeley
  • Papers
  • RefWorks Tagged
  • Ref Manager
  • RIS
  • Zotero
‍ Request Permissions
Share
Outcomes for patients with the same disease treated inside and outside of randomized trials: a systematic review and meta-analysis
Natasha Fernandes, Dianne Bryant, Lauren Griffith, Mohamed El-Rabbany, Nisha M. Fernandes, Crystal Kean, Jacquelyn Marsh, Siddhi Mathur, Rebecca Moyer, Clare J. Reade, John J. Riva, Lyndsay Somerville, Neera Bhatnagar
CMAJ Nov 2014, 186 (16) E596-E609; DOI: 10.1503/cmaj.131693
Digg logo Reddit logo Twitter logo Facebook logo Google logo Mendeley logo
  • Tweet Widget
  • Facebook Like
  • Google Plus One

Jump to section

  • Article
    • Abstract
    • Methods
    • Results
    • Interpretation
    • Acknowledgments
    • Footnotes
    • References
  • Figures & Tables
  • Related Content
  • Responses
  • Metrics
  • PDF

Related Articles

  • Highlights
  • PubMed
  • Google Scholar

Cited By...

  • Improved in-hospital outcomes and care for patients in stroke research: An observational study
  • Google Scholar

More in this TOC Section

  • Physician choices in pulmonary embolism testing
  • Diagnosis-wide analysis of COVID-19 complications: an exposure-crossover study
  • Symptoms associated with a positive result for a swab for SARS-CoV-2 infection among children in Alberta
Show more Research

Similar Articles

Collections

  • Topics
    • Research methods & statistics
    • Patient safety & quality improvement

Content

  • Current issue
  • Past issues
  • Collections
  • Sections
  • Blog
  • Podcasts
  • Alerts
  • RSS
  • Early releases

Information for

  • Advertisers
  • Authors
  • Reviewers
  • CMA Members
  • Media
  • Reprint requests
  • Subscribers

About

  • General Information
  • Journal staff
  • Editorial Board
  • Governance Council
  • Journal Oversight
  • Careers
  • Contact
  • Copyright and Permissions

Copyright 2021, Joule Inc. or its licensors. All rights reserved. ISSN 1488-2329 (e) 0820-3946 (p)

All editorial matter in CMAJ represents the opinions of the authors and not necessarily those of the Canadian Medical Association or its subsidiaries.

Powered by HighWire