Partnering with families of children with medical complexity to evaluate interventions

• Children with medical complexity have serious chronic conditions, which lead to severe functional limitations, high health care needs and high resource use.

• A strategy to coordinate care for children with medical complexity and their families in Ontario is being evaluated using a randomized clinical trial design in which the outcomes were chosen by patients, caregivers and their providers.

• Co-selection of outcomes by patients, caregivers and their clinicians requires pre-trial consensus but holds promise for improving the applicability and meaning of trial results.

A key goal in health care is to improve outcomes that matter most to patients.1 This is a poignant aim for a small but very important group: children with medical complexity and their caregivers. Children with medical complexity are defined by serious chronic conditions, functional limitations, increased health care needs and high resource use.2 Although their numbers are relatively small (< 1% of Canadian children), children with medical complexity have an impact on the health care system that is large and growing.2 Care for these children accounts for about a third of all child health spending in Canada. 3 Caregivers of children with medical complexity take on enormous responsibilities for their child’s care,4^–6 and high levels of emotional7 and financial stress.8

The Ontario Ministry of Health and Long-Term Care funded a Complex Care for Kids Ontario strategy implemented by the Provincial Council for Maternal and Child Health to improve care coordination for children with medical complexity. Our team was tasked with evaluation using a wait-list randomized controlled trial design.

Image courtesy of FatCamera/iStock

Evaluating an intervention with a randomized trial design is considered the gold standard because it allows for mitigation of various important sources of bias. However, methodologic rigour in design does not necessarily translate into meaningful results for children and their caregivers. Bias can be introduced into otherwise well-designed trials through inappropriate selection of outcomes and their measures. There is a tendency among investigators to select criteria for evaluation that are meaningful to us but not necessarily to our patients; as such, the study’s results are biased toward results of interest only to the researchers.

Less than 1% of primary or even secondary end points in registered clinical trials for children with chronic medical conditions include outcomes that are important or meaningful to patients.9 If patient-important outcomes are not included in the evaluation of an intervention, policy-makers will have limited information on which to base decisions about what patients value.

We hypothesized that engaging caregivers with clinicians in outcome selection and development of measurement tools, and then evaluating the outcomes they selected in the trial, would lead to research findings that would be more meaningful to caregivers. The idea was that if the caregivers and the clinicians selected outcomes together, study results could be interpreted and applied by both groups. The first step was collaborating with our policy-maker and community of practice to structure the evaluation based on caregiver- and clinician-selected outcomes.

The respective priorities of caregivers and clinicians were assessed through an online survey to allow for geographically diverse groups to provide input. Stratification in the statistical analysis ensured representativeness of perspectives within the sample. This approach also helped distinguish between universally important outcomes and potentially discordant outcomes between caregivers and clinicians. Next, we used the survey results to hone consensus work with a smaller group of caregivers and clinicians during an in-person meeting. The objective of the consensus meeting was to solidify the outcomes that were universally important to caregivers and providers alike. Finally, to distinguish between competing end points and to set realistic criteria for success, we assessed the perceptions of caregivers and clinicians for the degree to which they perceived the outcomes as amenable to change by improved care coordination.

Despite undertaking this large amount of preparatory work before implementation of the evaluation, the experience of our team thus far has been that the positives have exceedingly outweighed the negatives. Having caregivers and clinicians help to select study outcomes showed us two clear gaps in our abilities as investigators to measure what was important to patients. First, we saw that existing health and quality-of-life measures contained unsuitable content and measurement properties for children with medical complexity. Second, we found that measures for key outcomes were unavailable for our target population (e.g., satisfaction with feeding and medical technology). This opened up an opportunity to develop tools to measure these experiences in collaboration with caregivers. Future development in participatory research with caregivers will meet the objective of creating tools that are interpretable to researchers and caregivers, while providing results that can be implemented in health care settings and have meaning in patients’ lives.

Conducting a rigorous process to engage patients and clinicians in defining outcomes took substantial time. There was an inevitable tension between the need for evaluation to take place rapidly within time-limited fiscal cycles, and the need for careful and thoughtful incorporation of the voice of the patient and clinicians into the process. Early partnership with the Provincial Council for Maternal and Child Health, which is leading implementation, helped to mitigate this. Roll-out to some patients was neccessary before formal evaluation began.

By engaging stakeholders in the development of our evaluation plan, we sought to obtain information relevant to patients and caregivers. We hoped that this approach would boost investment of caregivers of children with medical complexity in the research process and thus make data collection smoother. However, we did not find this to be the case. Caregivers were required to complete lengthy surveys. Although most were keen to participate in the research process, many had more pressing needs than answering study questions. Collecting data at times that were most convenient for the child and caregiver (e.g., waiting rooms for health care appointments for some, at home for others) using a variety of platforms (mobile devices for some, pen and paper for others) helped, at least in part, to overcome challenges with data collection.

Structuring an evaluation of a trial according to outcomes prioritized completely by patients’ caregivers (and their clinicians) is a novel extension of patient partnership in research. We hope that working with patients to select trial outcomes will produce results that are meaningful in the lives of children with medical complexity and their caregivers.