Improving precision medicine using individual patient data from trials

Problems with current models of evidence synthesis

Facing a rapidly growing body of medical knowledge, physicians rely on high-level aggregate-based knowledge resources, such as online knowledge bases, meta-analyses and clinical practice guidelines. However, this results, by default, in clinicians treating individual patients as if they were members of a homogeneous group. Yet, any successful trial is likely to have some participants who do not do as well as the “average” participant; what was best for the group may actually have been harmful for those participants. The masking of between-participant variability by aggregate-level design and analysis complicates prediction of the utility and risk of applying a given intervention to a particular patient.2

A meta-analysis may conclude that an intervention’s net (mean) effect is negligible, which would justify avoiding that intervention. However, if considerable variability exists among the populations and results of the analyzed trials, this conclusion may not apply to an individual patient who differs from the average patient. Frequent use of restrictive eligibility criteria in trials may further compromise their relevance to a particular patient, because they result in trial populations that do not properly represent that patient. This may limit the external validity of those trials3 with regard to a target population or a specific patient.

Providing all patients with the best-on-average treatment is arguably more justifiable than not relying on evidence at all, but the interests of the individual patient may not be well-served. Precision medicine attempts to close the evidence gap between a generic and real patient.4 Indeed, differentially applying evidence to different patient populations is an established practice, notably in cancer and coronary heart disease therapy. Subgroup analysis of clinical trial populations is commonly used to support this strategy. However as subpopulations are typically characterized by categories of a single baseline characteristic (e.g., age range or sex), a patient may belong to multiple subgroups exhibiting discordant trends with regard to the effect of an intervention. Integrating subgroup analyses to support clinical decisions related to a particular patient is a challenging task.5 Although there may be extensive evidence on the optimal management of a particular condition, using this management plan for patients who have multiple comorbid conditions may not be well supported. Dörenkamp and colleagues term this “the maze of multimorbidity.”6

Physicians commonly use clinical judgment when adjusting evidence to the care of an individual patient based on the patient’s special characteristics and circumstances. Unfortunately, the evidence gap commonly reduces this to a subjective (and thus often biased) process.7

What are the challenges in similarity assessments?

Reuse of individual patient data from clinical trials in advanced similarity assessments is challenging both conceptually and practically. Combining multiple data types to assess similarity at the patient level as a whole is complicated. Only baseline characteristics of trial participants that are collected can be considered in the matching process, and those are typically a small subset of the parameters that could be used to characterize an individual. However, trials do collect a wide range of demographic (e.g., age, sex and race) and clinical (e.g., disease severity and comorbidities) features, as well as setting-related features (e.g., type of hospital and geographic location).

Characteristics that predict the intervention outcome should be included in the similarity assessment. Unfortunately, these remain largely unknown and may vary by disease, intervention and subject. Indeed, domain knowledge can help in selecting relevant characteristics. Based on a physician’s clinical knowledge and judgment, they may define, on a case-by-case basis, through an interactive computer program what features should be used to select similar patients. Although of questionable validity, this is what physicians currently (and often implicitly) do when assessing the overall similarity between a patient and a trial population. Appreciating that physicians’ limited knowledge may not be sufficient to form a valid and comprehensive list of relevant characteristics, an alternative approach would be to include all available characteristics in the similarity assessment.

Some features that are relevant to similarity assessment may be more important than others. In most cases, it would probably be impossible to assign differential, evidence-based weights to various features. As with feature selection, clinical judgment could be used to weigh feature importance or unweighted assessment could be applied.

The use of individual-level data, especially with new data types, raises privacy issues, because integration of multiple data types has the potential to be used to identify patients. One way to ensure privacy is to mask individual-level data by only allowing users to view aggregated characteristics of a group of similar patients. For example, in the Observatonal Health Data Sciences and Informatics program, 14 clinical queries are run on disparate participating clinical databases and return aggregate-level results. This enables researchers across institutions and states to leverage insights from the data of millions of patients without exposing patient identity.

In some cases, sufficient data are not available to guide the management of a patient with a rare disease or unusual circumstances. Similarity analysis could be used to collect such cases, and the data would be used by clinicians in the same manner as case reports.

Evaluation of information resulting from automated analysis of clinical trial data would require human judgment. Some factors are important when making clinical decisions that are very difficult to measure and report in clinical trials. Patient frailty, values and preferences; availability of support at home; physician experience and beliefs; and other considerations influence the translation of evidence to real-world decisions.

How can statistical inference be used to personalize clinical trials?

Careful statistical analytic practices are imperative to support inference from subgroup analysis.5^,15 A subgroup of similar patients will be smaller than the original population from which it is selected. Inevitably, the precision of estimates of intervention effect size will be affected. This is the case with both demographic and clinical characteristics including comorbidities.16 Certain populations, especially older adults, infants and children, sicker patients or those with comorbidities, are generally underrepresented in clinical trials.17^,18 Subgroups of trial participants having those characteristics are likely to be smaller and thus less informative. Multiple post-hoc analyses are associated with a risk of false-positive findings, requiring the use of smaller p-value cut-offs for statistical significance.18^,19 Considering that subgroups are smaller and thus have less power, reaching statistical significance when participants are drawn from a single clinical trial may be difficult. Nevertheless, under some circumstances (e.g., when a subgroup differs substantially from the general trial population), such analyses may produce more accurate estimates, which is why subgroup analyses are done. As with meta-analyses, pooling similar participants from across multiple comparable clinical trials may increase the subpopulation sample size and narrow down confidence intervals of the estimates of intervention effect.