Many remain unconvinced of the merit of patient-oriented research without evidence of its positive impact on research or health outcomes.
Patient-oriented research did not emerge from evidence showing its positive impact on research quality, health policies and services, or health outcomes; its origins were moral, ethical and political.
We found that there is no consistency of reporting and terminology for patient-oriented research, and that a direct causal link between patient-oriented research and health policy or services or the health outcomes is difficult, if not impossible, to prove with traditional research approaches.
Despite these challenges, we can and should work toward both measuring what is feasible to measure and building a framework for measuring health outcome impact.
We propose a framework for patient-oriented research that includes measuring value to patients, value to researchers, improvements to research processes, impact on policies and decision-making, impact on health outcomes, and contributions to social change in health research.
“Show me the evidence” is the frequent response of skeptics of patient-oriented research, or of those who are not convinced of the merit of this approach without evidence of its positive impact on research or health outcomes.
The advance of patient-oriented research is rooted primarily in moral, ethical and political arguments1–5 and social trends in health care that are breaking down the barriers between health professionals and patients.6 The moral argument is that patients have a fundamental right to be engaged in research, or that it is an ethical mandate that leads to better transparency and accountability.1,7–9 Politically, patient-oriented research introduces democratic ideals so that public funding for research is spent responsibly and serves the public interest,1,6 leading to the democratization of science and the research process.3,7,10 Patient-oriented research is also based on a social imperative of engaging those who are most affected by health care decisions; such studies aim to build on patients’ perspectives, needs and priorities.9
Patient-oriented research is conducted in collaboration with patients (including their families and informal caregivers), clinicians and decision-makers; focuses on priorities and outcomes that matter to patients; and ultimately aims to integrate research into policy and practice to improve health care outcomes.11 Most importantly, it extends the role of a patient beyond that of a research subject to engage patients as meaningful and active partners of the research team. This approach is reflective of a global movement away from a traditional and often paternalistic model of health care, toward such models as self-management programs, patient-centred care and participatory action research.10 It is supported in the United Kingdom by the National Institute for Health Research program INVOLVE, in the United States through the Patient-Centered Outcomes Research Institute (PCORI) and in Canada through the Strategy for Patient-Oriented Research (SPOR).
Given its origins, the patient-oriented research approach was not based on evidence that it would have a positive impact on the research quality, on health policies and services and, ultimately, on health outcomes, although there are clearly expectations in this regard. This lack of antecedent evidence of the impact of patient-oriented research before its launch may explain why many authors are still trying to prove its merit, while others are attempting to invalidate assumptions regarding the value of the approach.8
We conducted a review and narrative analysis of the literature (Box 1) on the impact of patient-oriented research to consider both what is known about its impact and how it should be appreciated.
Methodology used in this Analysis
We conducted an initial exploratory search for systematic reviews of patient-oriented research and uncovered 4 that related to impact.7,13–15 Using a snowballing technique and a database search, we also located a further 43 papers related to measuring or exploring the impact of patient-oriented research. See Supplementary Table 1 in Appendix 1, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.190237/-/DC1, for descriptions of those articles most relevant to the development of this paper.
For the narrative analysis, we used a process that included identifying key points, amalgamating these into a central list, and then thematically analyzing and synthesizing the evidence. This enabled us to establish a broad perspective of what is known about the impact of patient-oriented research, from which we were able to generate and analyze a number of observations. See Appendix 1 for a further description of our methodology.
The definition of “impact” that guided our inquiry is “an effect on, change or benefit to the economy, society, culture, public policy or services, health, the environment or quality of life, beyond academia.”12 Impact from patient-oriented research would therefore produce a measurable change to broader society or, more specifically, in health policy or services or the health outcomes and quality of life of patients.
What is the evidence of impact of patient-oriented research?
In our review and analysis of the articles, we found that most were focused solely on the patient engagement component of patient-oriented research. We also found that authors were unable to find robust evidence of the impact of patient-oriented research or patient engagement,2,5,7,12,13,16,17 with the exception of a recent mixed-methods study by PCORI, which concluded that its funded projects are “on the path towards desired impact of engagement in research.”18 Systematic reviewers also maintained that evaluation of patient-oriented research was greatly lacking, thus putting its integrity into question.9,16
Those articles that do make claims that patient-oriented research has an impact do not back these statements with any empirical evidence, and often qualify their statements about impact with words such as “potential,” “hypothesized” or “theoretical.”12,19 Esmail, Moor and Rein note that “while many papers theorize or propose intended impacts, very few empirical studies have applied or tested hypothesized impacts.”1
Some of the articles that alluded to a positive impact were actually about specific benefits to the patient who is engaged in the research, such as giving them a sense of empowerment, ownership and fulfillment.14,15,20 Even then, it was often the case that authors’ claims were backed not by comprehensive research, but rather by anecdotal evidence, brief conversations or solitary case studies.21–23
According to Fergusson and colleagues,17 the answers to basic questions about the value of patient-oriented research remain unanswered. Hamilton and colleagues2 concur, explaining that although patient-oriented research is said to improve the relevance and quality of health research, there are limited empirical data to prove this.
Two key themes emerged from our analysis to explain why there is a lack of empirical evidence of impact: There is no consistency of reporting and terminology for patient-oriented research, and a direct causal link between patient-oriented research and health policy or services or the health outcomes is difficult, if not impossible, to prove with more traditional research approaches.
Theme 1: inconsistent terminology and reporting
The first theme that emerged was the substantial inconsistency in the way the terms related to patient-oriented research are defined,13 and the even greater variability in labels being used to describe this research. For example, other terms for research that is patient-oriented include “patient-centred outcomes research,” “user involvement,” “patient and service user engagement,” “consumer engagement,” “community-based research,” “participatory research” and “patient and public involvement.” In particular, even though patient engagement is only one — albeit integral — element of patient-oriented research, the terms are often conflated. Also, there are no established indexing terms in bibliographic databases, limiting the potential for standardized reporting.24
There is consensus among scholars that we are in the midst of a culture shift in which we still lack the necessary reporting tools and volume of published research to conduct a valid assessment.25–27 Findings from systematic reviews were considered incomplete owing to inferior or variable reporting methods uncovered in the literature;17,28 it is often the case that papers on patient-oriented research contain inadequate descriptions of the approach used.29 Staniszewska and colleagues recognized this lack of consistency and noted that it “creates a fragmented evidence base making it difficult to draw together our collective understanding of what works, for whom, why and in what context.”28 However, as pointed out by Mockford and colleagues, “the absence of evidence does not indicate an absence of impact; rather it indicates inadequate reporting with a lack of valid and reliable tools to capture the impact.”13
Theme 2: feasibility of measuring impact
A common recurring theme was that there are several key reasons why it is difficult, if not impossible, to measure the impact of patient-oriented research.
For one, no consistent and rigorous framework to assess the effectiveness of patient-oriented research exists.12 Although there are a number of studies that measure perceived and self-reported impacts, and tools that identify what is important to report in patient-oriented research — such as the Guidance for Reporting Involvement of Patients and the Public (GRIPP2) tool28 or the Patient Engagement in Research (PEIR) framework2 — there are few valid and reliable evaluation tools that are based on observable impacts or that will capture evidence-based impact.16,28 One possible explanation for this is that we have not yet properly conceptualized the approach, defined its purpose or developed a theoretical model on which we can base impact assessments.1,2,10 Carroll and colleagues note that measuring patient-oriented research outcomes is “a difficult proposition given that it was not clear what these outcomes should be or what statistical methods should be used to generate results.”10
Most importantly, several authors1,13,30 explain that it is likely not possible to isolate patient-oriented research methods — especially the patient engagement component — as an influence on outcomes or impact. Too many other factors and variables — such as context, policy, people, resources, the purpose of the consultation and the culture of organizations and of individuals3 — might confound the results of an assessment. As noted by Esmail and colleagues, “longer-term outcomes typically have complex causes that are difficult to trace back to one research study, let alone the engagement of patients or other stakeholders in the research enterprise.”1 Even PCORI, which was created to promote patient-centred research, admits that the links are less apparent between patient-oriented research and a broader impact on more distal outcomes such as patient decisions, quality of life and health, public policy or health services.27
It is clear that several questions remain, and there is very little evidence about the impact of patient-oriented research on research quality and health outcomes.1,17,27
How can we build evidence of impact?
Considering the substantial difficulties in linking patient-oriented research to health outcomes, we might ask if it is necessary or worthwhile to continue to attempt to measure its impact, or if it is enough to believe that patient-oriented research is socially, politically and ideologically the “right thing to do.”
In answering this question, it is important to ensure we are not applying a higher standard to patient-oriented research than to other health research. The Canadian Academy of Health Sciences, in its report31 on a proposed impact framework for health research, acknowledges that making the linkages from any funded research to health outcomes is a major challenge, even though “it is precisely these outcomes that are the foundation for the public’s support of health research.” For patient-oriented research, these outcomes are the foundation to convincing health researchers and patients of its merit.
We believe that, even though it might be challenging and still too soon to effectively show long-term outcomes and the overall success of patient-oriented research,3 we can and should work toward this goal by measuring what is feasible to measure, and building a framework for measuring health outcome impact.
Figure 1 outlines a potential framework for measuring various levels of outcomes from patient-oriented research that build toward providing evidence of impact. In the figure, boxes 1–3 outline outcomes that are relatively easy to measure, as it is possible to establish the causal links between a patient-oriented research approach and these more direct outcomes. These include benefits and advantages to the patient partners, researchers and to the research itself. As more patient-oriented research studies are published and reporting becomes more consistent, we will be better able to determine its broader impact with measures such as those in boxes 4 and 5.
The proposed measures presented in Figure 1, boxes 1–5, are drawn from the Canadian Academy of Health Sciences framework,31 inspired by a community-based participatory research logic model,32 and adapted from the protocol recently developed by Graham and colleagues33 in the area of integrated knowledge translation. The Canadian Academy of Health Sciences framework outlines indicators in areas such as advancing knowledge, informing decision-making, health impacts, and broad economic and social impacts. The community-based participatory research logic model includes assessments of impact and outcomes relevant to patient-oriented research, such as research design, system and capacity changes, and improved health. The goals of the integrated knowledge translation protocol include testing assumptions about its ability to lead to more relevant, useful and applicable research that results in greater impact; to increase the use of research in practice, health systems and policy decisions; and to improve patient and health system outcomes.33 Because knowledge translation is a key component of patient-oriented research and the goals of integrated knowledge translation overlap substantially with this approach, the protocol from Graham and colleagues33 might be easily adapted or formulated for patient-oriented research.
We might also aim to measure the impact of the contribution of patient-oriented research on society34 (Figure 1, box 6). In other words, if patient-oriented research is considered ethically, morally and politically “the right thing to do” and endeavours to break down the barriers between health professionals and patients,6 then we might make it a priority to ask questions such as, 1) Is patient-oriented research contributing to a positive and transformational change in our research culture and behaviours? and 2) Are the values and principles behind this approach becoming embedded in health research, creating social change and the redistribution of power at a macro level?9
Building consensus
An important component of the work we do to secure a place for patient-oriented research in the future of health research is to develop conceptual guidance and gain consensus internationally on what it means to conduct this research.1 This might be accomplished through an international forum on patient-oriented research that leverages the momentum of this field to develop a document that outlines such elements as a conceptualization of the approach, a theoretical model, definition of a clear purpose, standardized terminology and reporting, and consensus on a framework for measuring impact. This forum would be an opportunity to consolidate learning to advance the art and science of patient-oriented research.
Conclusion
Embracing patient-oriented research is a socially motivated approach that requires a profound culture shift and, for some researchers, represents a difficult transition from what they deem to be more traditional approaches. This shift will require “shaking up existing research hierarchies, not merely smoothing out a few bumps in current practice.”35
There will always be skeptics who avoid conducting patient-oriented research because they want definitive proof of its impact on health outcomes. Patient-oriented research is a social imperative, it is ethically the “right thing to do” and it represents another step forward in the journey to include patients in their own health care and to move research into practice. It is also the right thing to do to persist in collective efforts to measure its impact effectively, and to respond to skeptics’ cries of “show me the evidence!”
Footnotes
Competing interests: All authors work for the Alberta Strategy for Patient-Oriented Research (SPOR) SUPPORT Unit. No other competing interests were declared.
This article has been peer reviewed.
Contributors: Dean Eurich and Diane Aubin conceived of and designed the work. Diane Aubin acquired and analyzed the data; all authors interpreted the data. Diane Aubin drafted the manuscript and Dean Eurich and Marilynne Hebert revised it critically for intellectual content. All authors gave final approval of the version to be published and agreed to be accountable for all aspects of the work.
Funding: The authors receive funding for provincial SPOR activities in the province, both for operations and in salary (Aubin), and stipend research support (Eurich and Hebert) for managing the unit.
References
In this issue
Article tools
Jump to section
Related Articles
Cited By...
- Changing relationships: how does patient involvement transform professional identity? An ethnographic study
- Partnering with patients in the production of evidence
- Developing a program theory of patient engagement in patient-oriented research and the impacts on the health care system: protocol for a rapid realist review
- Patient-oriented research can be meaningful for clinicians and trialists as well as patients
More in this TOC Section
Similar Articles
Collections
