Explaining pragmatic trials to pragmatic policy-makers ====================================================== * Malcolm Maclure Successful policy-makers value pragmatism. As politics is “the art of the possible,” pragmatism is the art of the practical and workable. It entails getting more results sooner through flexibility rather than slavish adherence to rigid preconceptions. This requires experience with the trade-offs between quality and timeliness, between central control and local adaptation, and between leading and following. It calls for good judgment on when to uphold principles versus when to compromise, such as when to abide by experts’ systematic reviews of evidence versus heeding opinion-based consensus. The term “pragmatic randomized trials,” therefore, is intrinsically attractive to policy-makers. Juxtaposed with “pragmatic,” the term “explanatory” suggests something more advisory. In the culture of policy-making, the juxtaposition is reminiscent of the 2 major types of briefing notes: “for decision” and “for information.” The first serves decision-making on specific actions such as funding, regulations and organizational changes. The second serves communication of news, history, context, long-term options and strategic directions. In this commentary, my purpose is pragmatic: to help researchers use the article by Thorpe and colleagues 1 (see page [E47](http://www.cmaj.ca/lookup/volpage/180/E47?iss=10) of this issue of *CMAJ*) as well as 2 articles published in the May 2009 issue of the *Journal of Clinical Epidemiology*2,3 to promote pragmatic trials with language comfortable to policy-makers. First I will recount how each article influenced my thinking. Then I will present several pictures to help policy-makers grasp the issues. I will conclude with recommendations. I was introduced to the concepts presented in the article by Thorpe and colleagues on the pragmatic–explanatory continuum indicator summary (PRECIS) tool at a workshop on pragmatic trials in 2005. I arrived at the workshop disliking the term “pragmatic.” To my mind, it suggested expedience is more important than methodological rigour. Although I had often opted for expedience when promoting policy trials, I was not proud of my compromises. My mind was changed by the workshop, the PRECIS paper and the original paper by Schwartz and Lellouch on the distinction between explanatory and pragmatic trials. 4 I realized the trialists’ definition of “pragmatic” echoed the empirical philosophy of pragmatism originated by the scientist Charles Peirce. 5 He argued that the importance of an idea or action lies in whether it makes a difference in everyday life. Ideas or actions that correspond to attractive explanations (e.g., metaphysical theories), but make no difference to outcomes, are problematic. I was invited to the workshop to present lessons from our policy trials in British Columbia, which we call “designed delay trials” (DDTs). Until then I had not reflected systematically about the dimensions of our trials. The PRECIS paper by Thorpe and colleagues helped me understand that designed delayed trials are pragmatic in 2 senses of the word: their purpose and design are to inform policy decisions (the technical meaning of “pragmatic” used by trialists), and they involve compromises for expedience (the lay meaning of “pragmatic” used by policy-makers that rigorous researchers often disdain). The PRECIS paper’s systematic approach made me self-conscious of my compromises for expediency in the pursuit of randomized delayed control groups. I worried that designed delayed trials could tarnish the worthy term “pragmatic trial.” When I was invited to write a paper on designed delayed trials, I decided to contrast them with rigorous pragmatic trials. 6 Only after writing the first draft did I realize the distinctive feature of our designed delayed trials is ongoing negotiation between the policy-maker and the researcher about the existence, size, duration and definition of the control group. Because policy-makers are accountable for their policies, they retain the power to whittle down or veto any suggestions by researchers. I had to admit I had become a closet compromiser, willing to abandon some principles of rigorous methodology. I justified this by saying randomization is such an improvement over nonrandomized control groups, many other flaws in a study are tolerable prices to pay. The workshop and the PRECIS paper brought me out of the closet, able to reflect on the causes and effects, the risks and benefits, of such compromises. The second article, by Karanicolas and colleagues, 2 defining mechanistic and practical trials, made me appreciate in a new way how clinicians’ world view is dichotomized between mechanistic knowledge (biological and pathophysiological information taught in medical school) versus the empiricism of clinical practice. Clinicians’ personal experience of that dichotomy is a solid initial foundation for explaining the distinction between explanatory and pragmatic trials. For them, the words “mechanistic” and “practical” vividly bring to mind the 2 major forces influencing their clinical decisions. Although I accept the merits of the mechanistic–practical dichotomy as a starting point to explain pragmatic trials to practising clinicians, I feel it should not be the end point. What I most appreciate about their article is how it opened my eyes to the need to explain pragmatic trials to clinicians in clinical terms and policy-makers in policy terms. What is practical for one clinician and one patient when deciding on one treatment may differ greatly from what is pragmatic for a policy-maker when deciding on a policy for a population. Consider a randomized double-blind n-of-1 trial that shows drug A works better than placebo in a particular patient, Jane Doe. Such a clinical trial is practical both for that patient and the prescriber, yet tells us nothing about the drug’s mechanism. Drug A is an explanation for the patient’s improvement, but Jane Doe’s response is not a sound basis for the clinician to change his or her personal clinical policy toward all patients with the same diagnosis. Such an n-of-1 trial is explanatory, not pragmatic, yet it is practical, not mechanistic. The terms “explanatory” and “pragmatic” are superior for the reasons given by Oxman and colleagues. 3 Policy-makers need terms and definitions that apply to nonbiological causes and correspond with the major forces influencing policy decisions. When Oxman and colleagues refer to conflicts of interest, they hint at an issue not mentioned in the other articles: we researchers are sharpening methodology tools that will be used not only by grant agencies and editors, but also by marketers, lobbyists and lawyers. Nonrandomized and uncontrolled n-of-1 experiments — in other words, anecdotes from *practical* clinical experience — are widely used by advocates of new treatments. So are selective positive explanatory trials. In the face of such biased advocacy, pragmatic trials offer a scientific method of response for policy-makers, including clinicians (who are themselves policy-makers when they set the patterns of their own clinical practices). As a result of these papers, I would explain pragmatic trials to policy-makers as follows: A pragmatic trial is a real-world test in a real-world population, whereas an explanatory trial is a specialized experiment in a specialized population. The differences are shown at a glance in Figure 1 and Figure 2. Figure 1 shows the major dimensions of a trial that influence its *applicability* as the trial moves from its purpose to its result. The dimensions can be divided between the provider of the new intervention and its target recipient who normally receives standard care. Figure 2 shows the same image incorporating real-world variation in types of providers and recipients, as well as interventions, follow-up procedures and analyses. The contrast between the 2 figures portrays the distinction between explanatory and pragmatic trials. Explanatory trials aim to demonstrate that an intervention *can* work in special circumstances — with specialized clinicians and interventions, and selected patients and outcomes. Pragmatic trials aim to test whether an intervention *does* work in real-world general circumstances, more applicable to the policy-maker’s purposes and settings. ![Figure1](http://www.cmaj.ca/https://www.cmaj.ca/content/cmaj/180/10/1001/F1.medium.gif) [Figure1](http://www.cmaj.ca/content/180/10/1001/F1) **Figure 1:** Ten major dimensions of an explanatory (intervention) trial 1 that influence the trial’s *applicability* as it moves from its purpose to its result (from left to right). The dimensions can be divided between the provider of the intervention (e.g., clinician) and its target recipient who normally receives standard care (e.g., patient). See Figure 3 for examples of other providers and recipients. ![Figure2](http://www.cmaj.ca/https://www.cmaj.ca/content/cmaj/180/10/1001/F2.medium.gif) [Figure2](http://www.cmaj.ca/content/180/10/1001/F2) **Figure 2:** Ten major dimensions of a pragmatic trial 1 showing real-world variation. In contrast to the restrictions on these dimensions in explanatory trials, as illustrated in Figure 1, pragmatic trials incorporate greater diversity in selection of providers and recipients, flexibility of the intervention, degrees of adherence, scrutiny of participants, and types of outcomes and analyses. Pragmatic trials are more likely to have multiple purposes, addressed by multiple analyses. It should be stressed that providers need not be clinicians and recipients need not be patients. Often the cause and effect are at another pair of levels in the health system hierarchy, as shown in Figure 3. For example, the providers might be regional governments and the recipients might be institutions, or the providers might be institutions and the recipients might be physicians. ![Figure3](http://www.cmaj.ca/https://www.cmaj.ca/content/cmaj/180/10/1001/F3.medium.gif) [Figure3](http://www.cmaj.ca/content/180/10/1001/F3) **Figure 3:** The providers need not be clinicians and the recipients need not be patients. Often the units of intervention and outcome analysis — like the cause and the effects — are at other levels in the health system hierarchy. In conclusion, an exact distinction between “pragmatic” and “explanatory” may be as elusive as an exact distinction between qualitative and quantitative research. Pragmatic trials can be characterized by numerous contrasts with explanatory trials, but the 2 types of trials should be defined in simple approximate terms (e.g., the trial’s purpose) that can be used orally by lay people in committee meetings. For policy-makers, I suggest saying that pragmatic trials are real-world studies “for decision,” whereas explanatory trials are specialized studies “for information.” See related article by Thorpe and colleagues, page [E47](http://www.cmaj.ca/lookup/volpage/180/E47?iss=10), and commentary by Zwarenstein and Treweek, page [998](http://www.cmaj.ca/lookup/volpage/180/998?iss=10) ## Footnotes * Published at [www.cmaj.ca](http://www.cmaj.ca) on Apr. 16, 2009. This article was published simultaneously in the May 2009 issue of the *Journal of Clinical Epidemiology* ([www.jclinepi.com](http://www.jclinepi.com)). **Competing interests:** None declared. ## REFERENCES 1. 1. Thorpe KE, Zwarenstein M, Oxman AD, et al. A pragmatic–explanatory continuum indicator summary (PRECIS): a tool to help trial designers. *J Clin Epidemiol* 2009;62:464–75. [Also in CMAJ 2009;180:E47–E57.] [FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiRlVMTCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czoxMDoiMTgwLzEwL0U0NyI7czo0OiJhdG9tIjtzOjIyOiIvY21hai8xODAvMTAvMTAwMS5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 2. 2. Karanicolas PJ, Montori VM, Deveraux PJ, et al. A new “Mechanistic–Practical” framework for designing and interpreting randomized trials. J Clin Epidemiol 2009;62:479–84. [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.jclinepi.2008.02.009&link_type=DOI) [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=18468856&link_type=MED&atom=%2Fcmaj%2F180%2F10%2F1001.atom) [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000265424400004&link_type=ISI) 3. 3. Oxman AD, Lombard C, Treweek S, et al. Why we will remain pragmatists: four problems with the impractical mechanistic framework and a better solution. J Clin Epidemiol 2009;62:485–8. [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.jclinepi.2008.08.015&link_type=DOI) [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=19348973&link_type=MED&atom=%2Fcmaj%2F180%2F10%2F1001.atom) 4. 4. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. *J Chronic Dis* 1967;20:637–48. [Reprinted in J Clin Epidemiol 2009;62:499–505.] 5. 5. Wiener PP, editor. Pragmatism. In:*The dictionary of the history of ideas: studies of selected pivotal ideas*. Vol 3. New York (NY): Charles Scribner’s Sons; 1973. p. 551–70. 6. 6. Maclure M, Carleton B, Schneeweiss S. Designed delays versus rigorous pragmatic trials: Lower carat gold standards can produce relevant drug evaluations. Med Care 2007;45:S44–9. [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1097/MLR.0b013e318068932a&link_type=DOI) [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=17909382&link_type=MED&atom=%2Fcmaj%2F180%2F10%2F1001.atom)