The authors respond to “Assessing methodologic quality of a meta-analysis on managing chronic disease in older adults”

We thank Dr. Bidhendi Yarandi for the letter about our CMAJ article on the effectiveness of interventions for managing multiple high-burden chronic diseases in older adults.1^,2 We appreciate the interest in our systematic review and the assessment of its methodological quality using the AMSTAR 2 (Assessing the Methodological Quality of Systematic Reviews) appraisal tool.3 The items identified pertain to details of the systematic review methods that were not included in our published manuscript because of journal constraints on article length, so we welcome the opportunity to share these details in this letter. We disagree with Dr. Bidhendi Yarandi’s conclusion that our study is low quality.

First, conducting a systematic review of complex interventions is challenging given the expected heterogeneity across studies. This is even more challenging for complex interventions that integrate the care of multiple chronic conditions. We are the first to identify and make sense of multimorbidity interventions that were deliberately created for older adults with a particular combination of disease dyads or triads (rather than a single disease). As such, we worked very hard to assess all of the clinical (variability in participants, interventions and outcomes), methodological (variability in the study design and risk of bias) and statistical (variability in intervention effects) heterogeneity. 4 This included an innovative assessment to determine which intervention components or combination of components contributed to the observed heterogeneity and their impact for which specific disease dyads or triads and for which outcomes. Such an extensive examination of heterogeneity allowed us to decide which studies were deemed the most appropriate to pool in a meta-analysis, which is rarely done.

Not surprisingly, we identified substantial heterogeneity among our 21 included trials, which varied widely in their disease combinations, intervention components and outcomes. We identified 8 trials as appropriate to contribute to our meta-analyses across our primary and secondary outcomes. However, most of these meta-analyses included 2 studies only,2 which precluded any further assessment of publication bias or effect of risk of bias. Additionally, most of our studies had low risk of bias for most factors,2 making any further subgroup analyses redundant.

Lastly, although reporting findings using prediction intervals in a random effects model has been recently suggested to enhance the clinical meaning of findings, 5 we did not consider it because it’s not specifically recommended by the Cochrane Handbook for Systematic Reviews of Interventions4 and the estimate is imprecise if based on only a few studies.5

Overall, our systematic review was rigorously planned and executed as per Cochrane standards.4 Additionally, we advanced the knowledge of conducting a rigorous systematic review of complex interventions by applying an intervention deconstruction strategy that allowed us to generate a more appropriate and meaningful estimation of impact from our meta-analyses.