 Advertisement | ||||||||
 | ||||||||
|
||||||||
 |
 |
 |
 |
 |
 |
 |
||
|
Dr. Montori is with the Division of Endocrinology, Metabolism, and Nutrition, Department of Internal Medicine, Mayo Clinic and Foundation, Rochester, Minn. Dr. Guyatt is with the Departments of Medicine and of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ont.
Correspondence to: Dr. Gordon H. Guyatt, McMaster University Health Sciences Centre, 1200 Main St. W, Hamilton ON L8N 3Z5; fax 905 577-0017; guyatt{at}McMaster.ca
In deciding if a treatment is likely to work in an individual patient, clinicians need to know the effect of the intervention in patients who take the treatment as prescribed. However, participants in clinical trials may not adhere to the protocol, or clinicians may recommend withdrawal of the study medication because of apparent adverse effects. Should investigators exclude from the analysis any participants who violate the research protocol? In this article, we review how randomization reduces bias in clinical trials and then discuss the importance of including all eligible patients in the analysis, to ensure the validity of the results.
Preventing bias in randomized controlled trials
The randomized controlled clinical trial is the best way to minimize bias in ascertaining treatment effects. The intent of randomization is to establish groups of patients with similar distributions of the characteristics that could determine whether they will suffer the adverse outcome of interest. If prognostic factors are balanced in the 2 (or more) groups and if the treatment has no effect, the proportion of participants experiencing the target outcome will be similar in the arms of the study. Conversely, if differences in outcome are observed, clinicians can confidently attribute those differences to the experimental intervention.
Applying the intention-to-treat principle
How should investigators analyze study data if one or more patients have not adhered to the allocated management strategy, for whatever reason? Some investigators deal with these protocol violations by excluding the participants from the analysis. This form of analysis, known as a per protocol, efficacy, explanatory analysis, or analysis by treatment administered, describes the outcomes of the participants who adhered to the research protocol. Although investigators can use information from such an analysis to estimate the intervention's efficacy in those who actually received it in the intended intensity or dose for the intended interval, this estimate is likely to be seriously flawed.
The problem arises because the reasons for nonadherence to the protocol may be related to prognosis. Empirical evidence suggests that participants who adhere tend to do better than those who do not adhere, even after adjustment for all known prognostic factors and irrespective of assignment to active treatment or placebo.1,2 Excluding nonadherent participants from the analysis leaves those who may be destined to have a better outcome and destroys the unbiased comparison afforded by randomization.3
A hypothetical example will illustrate how excluding patients who do not receive the treatment to which they are assigned can introduce bias. Imagine a randomized trial of 200 patients with cerebrovascular disease, of whom 100 are assigned to receive acetylsalicylic acid (ASA) and a surgical intervention for which there is a 1-month waiting period and the other 100 are assigned to receive ASA alone (Fig. 1). Let us assume that the surgery is ineffective in preventing stroke — that is, on average, the same proportion of patients in each of the 2 arms (surgery + ASA and ASA only) will suffer a stroke.
|
In the surgery + ASA arm, 10 of the 100 patients have a stroke, the primary outcome of the trial, in the 1-month waiting period between randomization and surgery. Of the 90 patients who go on to have the surgery, 10 have a stroke in the subsequent year. Because in the absence of an effective intervention the patients in the ASA-only arm share the same destiny as those in the surgery + ASA arm, and because we have assumed that surgery is ineffective, we know that 10 patients in the ASA-only arm will have a stroke in the month after randomization and another 10 will do so in the subsequent year.
If we restrict the analysis in the surgery + ASA arm to the patients who underwent surgery (a per protocol analysis), the event rate would be 11% (10/90); however, the rate in the ASA-only arm would be 20% (20/100). These values represent a spurious (since we have assumed that surgery in fact has no effect on subsequent occurrence of stroke) reduction in stroke risk of close to 50%. Alternatively, if we count all events in all randomized patients, according to the intention-to-treat principle, we find that there were 20 events in each group and no evidence of a positive treatment effect. This analysis eliminates the misleading estimate of surgery's impact that is obtained with the per protocol analysis.
As illustrated by this example, applying the intention-to-treat principle provides an unbiased assessment of the efficacy of the intervention at the level of adherence observed in the trial. This level of adherence could be similar to that observed in the community, and the results could inform community-based decisions about the effectiveness of the experimental intervention.
However, clinicians make decisions at the level of the individual patient. Patients considering a treatment regimen and committed to complying with treatment wish to know how well the treatment will work when they use it at the intended dose for the intended duration. We have seen how per protocol analysis fails to answer this question and introduces bias. Unfortunately, applying the intention-to-treat principle doesn't solve the problem. When treatment is effective but nonadherence is substantial, the analysis following the intention-to-treat principle underestimates the magnitude of the treatment effect that will occur in adherent patients.
The solution involves conducting an intention-to-treat analysis, but only after using a protocol that ensures maximal adherence. For instance, studies may include run-in periods that identify nonadherent participants or participants intolerant to the experimental treatment, so they can be excluded before randomization.4
Dealing with loss to follow-up
Intention-to-treat analysis cannot minimize bias introduced by loss to follow-up, that is, patients whose outcome status is unknown. If investigators stop following patients who do not adhere to the study protocol, they will be unaware if those patients suffered the target outcome. If they conduct an intention-to-treat analysis and count events in all participants of whose outcomes they are aware (that is, those who followed the protocol), they will, de facto, be conducting a per protocol analysis.
Investigators often include patients lost to follow-up in the denominators in tables describing study results and in calculating estimates of effect. This approach assumes that none of those lost to follow-up suffered the target outcome. Making this unlikely assumption opens the door to a misleading presentation of study results. Alternative strategies are available that impute outcomes to those lost to follow-up. Some of these strategies include using multivariate analysis of prognostic factors to predict the most likely outcome in those lost to follow-up, imputation of outcomes by carrying the last known outcome status forward and analysis of best-case and worst-case scenarios. Nonetheless, these strategies in general make unverifiable assumptions that may introduce bias in the estimates of treatment effect.5 Thus, inferences from studies with appreciable loss to follow-up are usually weaker.
Determining if intention-to-treat analysis was used
Clinicians evaluating a randomized trial need to know if the researchers applied the intention-to-treat principle. A quick approach is to scan the Methods section of the published report looking for the phrase "intention-to-treat analysis." Two surveys of randomized controlled trials published in major medical journals during 1993–19956 and 19975 found that half of the reports used the term "intention-to-treat analysis." Unfortunately, the term was not always used appropriately. Thus, readers must look carefully at what was actually done, rather than looking only for the term.
In particular, significant loss to follow-up may introduce exactly the same bias as a per protocol analysis. For instance, Silverstein and associates7 reported the results of a trial in which 8843 patients taking nonsteroidal anti-inflammatory agents for rheumatoid arthritis were randomly assigned to receive misoprostol (4404 patients) or placebo (4439 patients) to prevent gastroduodenal complications, as judged by outcome assessors blinded to treatment assignment. The authors described their analysis as an intention-to-treat analysis. However, patients lost to follow-up were included in the denominator of event rates. Inclusion of these patients in the denominator without accounting for their outcomes in the numerator assumes that no patient lost to follow-up had gastroduodenal complications. The size of the groups lost to follow-up (1851 patients in the misoprostol group and 1617 in the placebo group) eclipsed the number of patients who experienced the primary end point in each group (25 in the misoprostol group and 42 in the placebo group), a situation that leaves the reader uncertain about the true magnitude of the treatment effect.
In another example, Harris and associates8 reported the results of study in which 1628 postmenopausal women with a previous vertebral fracture were randomly assigned to receive risedronate (813 patients) or placebo (815 patients) to prevent another vertebral fracture, as judged by a radiologist blinded to treatment assignment. These authors also described their analysis as an intention-to-treat analysis. After 3 years, 324 patients in the risedronate arm and 365 patients in the placebo arm had been lost to follow-up. The authors reported outcomes up to the point of last follow-up (using survival analysis), including 61 in the risedronate group and 93 patients in the placebo group with new vertebral fractures; the relative risk reduction was 41% in favour of risedronate. Those lost to follow-up from the placebo group were at higher risk (had more vertebral fractures) at baseline than either those in the placebo group who completed the study or those lost from the risedronate arm. This indicates that the placebo patients remaining in the study were a good-prognosis group and had, on average, a better prognosis than the remaining patients in the risedronate group. Because the risedronate group experienced fewer vertebral fractures than the placebo group, the substantial loss to follow-up in this case does not weaken the inference that risedronate results in a relative risk reduction of about 41%.
Conclusions
If randomized controlled trials are to provide unbiased assessments of treatment efficacy, investigators must apply the intention-to-treat principle. To improve the applicability of study results to individual patients, investigators should improve study design to ensure protocol adherence with minimal loss to follow-up. Finally, loss to follow-up can result in exactly the same sort of bias as a per protocol analysis. Therefore, if there is significant loss to follow-up, statements that investigators conducted an "intention-to-treat analysis" generally provide little reassurance.
Acknowledgments
This work was supported in part by the American Medical Association.
Footnotes
This article has been peer reviewed.
Competing interests: None declared.
References
This article has been cited by other articles:
|
|
 |
|
  P. T. L. Hoa, N. T. Huy, L. T. Thu, C. N. Nga, K. Nakao, K. Eguchi, N. H. Chi, B. H. Hoang, and K. Hirayama Randomized Controlled Study Investigating Viral Suppression and Serological Response following Pre-S1/Pre-S2/S Vaccine Therapy Combined with Lamivudine Treatment in HBeAg-Positive Patients with Chronic Hepatitis B Antimicrob. Agents Chemother., December 1, 2009; 53(12): 5134 - 5140. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S. Raymond, I. Favilla, and L. Santamaria Comparing Ultrasound Biometry with Partial Coherence Interferometry for Intraocular Lens Power Calculations: A Randomized Study Invest. Ophthalmol. Vis. Sci., June 1, 2009; 50(6): 2547 - 2552. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 R. Ziegler Mistletoe Preparation Iscador: Are there Methodological Concerns with Respect to Controlled Clinical Trials? Evid. Based Complement. Altern. Med., March 1, 2009; 6(1): 19 - 30. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 D. M. Hartung and D. Touchette Overview of clinical research design Am. J. Health Syst. Pharm., February 15, 2009; 66(4): 398 - 408. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. P. Rios, A. Odueyungbo, M. O. Moitri, M. O. Rahman, and L. Thabane Quality of Reporting of Randomized Controlled Trials in General Endocrinology Literature J. Clin. Endocrinol. Metab., October 1, 2008; 93(10): 3810 - 3816. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 Y-Y Chen and Y-C Chen Evaluating ethics consultation: randomised controlled trial is not the right tool J. Med. Ethics, August 1, 2008; 34(8): 594 - 597. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 P. J. Karanicolas, M. Bhandari, B. Taromi, E. A. Akl, D. Bassler, P. Alonso-Coello, D. Rigau, D. Bryant, S. E. Smith, S. D. Walter, et al. Blinding of Outcomes in Trials of Orthopaedic Trauma: An Opportunity to Enhance the Validity of Clinical Trials J. Bone Joint Surg. Am., May 1, 2008; 90(5): 1026 - 1033. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. Reddy, S. S. Gill, S. R. Kalkar, W. Wu, P. J. Anderson, and P. A. Rochon Oral Drug Therapy for Multiple Neglected Tropical Diseases: A Systematic Review JAMA, October 24, 2007; 298(16): 1911 - 1924. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 A. I Greenspan, S. L Wolf, M. E Kelley, and M. O'Grady Tai Chi and Perceived Health Status in Older Adults Who Are Transitionally Frail: A Randomized Controlled Trial Physical Therapy, May 1, 2007; 87(5): 525 - 535. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 K. Whittaker, C. Sutton, and C. Burton Pragmatic randomised controlled trials in parenting research: the issue of intention to treat. J Epidemiol Community Health, October 1, 2006; 60(10): 858 - 864. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
V. K. Bubbar and H. J. Kreder The Intention-to-Treat Principle: A Primer for the Orthopaedic Surgeon J. Bone Joint Surg. Am., September 1, 2006; 88(9): 2097 - 2099. [Full Text] [PDF]
|
|
|
|
 |
|
 V. M. Montori, Y. G. Wang, P. Alonso-Coello, and S. Bhagra Systematic Evaluation of the Quality of Randomized Controlled Trials in Diabetes Diabetes Care, August 1, 2006; 29(8): 1833 - 1838. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 W.-T. Chien, S. W. C. Chan, and D. R. Thompson Effects of a mutual support group for families of Chinese people with schizophrenia: 18-month follow-up The British Journal of Psychiatry, July 1, 2006; 189(1): 41 - 49. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 G Wright, R L Manser, G Byrnes, D Hart, and D A Campbell Surgery for non-small cell lung cancer: systematic review and meta-analysis of randomised controlled trials Thorax, July 1, 2006; 61(7): 597 - 603. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 V Jelic, M Kivipelto, and B Winblad Clinical trials in mild cognitive impairment: lessons for the future J. Neurol. Neurosurg. Psychiatry, April 1, 2006; 77(4): 429 - 438. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 L. L. Gluud Bias in Clinical Intervention Research Am. J. Epidemiol., March 15, 2006; 163(6): 493 - 501. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 M. Brell, A. Tortosa, E. Verger, J. M. Gil, N. Vinolas, S. Villa, J. J. Acebes, L. Caral, T. Pujol, I. Ferrer, et al. Prognostic Significance of O6-Methylguanine-DNA Methyltransferase Determined by Promoter Hypermethylation and Immunohistochemical Expression in Anaplastic Gliomas Clin. Cancer Res., July 15, 2005; 11(14): 5167 - 5174. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 J. H. Powers, D. Lin, D. Ross, R. Wunderink, M. Kollef, and J. Rello FDA Evaluation of Antimicrobials: Subgroup Analysis Chest, June 1, 2005; 127(6): 2298 - 2301. [Full Text] [PDF]
|
|
|
|
 |
|
 D. Brooks, K. O'Brien, E L. Geddes, J. Crowe, and W D. Reid Is inspiratory muscle training effective for individuals with cervical spinal cord injury? A qualitative systematic review Clinical Rehabilitation, March 1, 2005; 19(3): 237 - 246. [Abstract] [PDF]
|
|
|
|
 |
|
 I. D. Jackson, B. H. Heidemann, J. Wilson, I. Power, and R. D. Brown Double-blind, randomized, placebo-controlled trial comparing rofecoxib with dexketoprofen trometamol in surgical dentistry Br. J. Anaesth., May 1, 2004; 92(5): 675 - 680. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| HOME | CURRENT ISSUE | PAST ISSUES | COLLECTIONS | HELP | SEARCH |
|
|||||
|
Copyright 1995-2001, Canadian Medical Association. All rights reserved. ISSN 1488-2329
(e) 0820-3946 (p)
All editorial matter in CMAJ represents the opinions of the authors and not necessarily those of the Canadian Medical Association. |
|||||
