Review Article
New methods can extend the use of minimal important difference units in meta-analyses of continuous outcome measures

https://doi.org/10.1016/j.jclinepi.2012.02.008Get rights and content

Abstract

Objective

For continuous outcomes measured using instruments with an established minimally important difference (MID), pooled estimates can be usefully reported in MID units. Approaches suggested thus far omit studies that used instruments without an established MID. We describe an approach that addresses this limitation.

Study Design

Using the ratio of MID to standard deviation in the trials with an established MID, we imputed the MID for instruments without an established MID and pooled across all trials. We applied this approach to two meta-analyses.

Results

In 20 trials of respiratory rehabilitation, the pooled estimate did not differ significantly between trials with an established MID and those without an established MID (interaction P = 0.23). The same was true for 52 trials examining amitriptyline vs. other antidepressants (interaction P = 0.54). In the respiratory example, the addition of trials without an established MID led to little change in point estimates or confidence intervals (CIs, more data balanced by more heterogeneity in a random effects model). In the antidepressant example, the additional trials resulted in an identical point estimate with a narrowing of the CI.

Conclusion

Our method allows estimates of a pooled effect in MID units using both trials with and without an established MID.

Introduction

What is new?

  • To improve the interpretability of continuous data, we have previously described a method for pooling randomized trial data in minimally important difference (MID) units.

  • Many instruments, however, do not have an established MID, and our method thus far omits these studies.

  • Using the standard deviation ratio method described here, we have generated pooled estimates in MID units using all available data.

  • The approach minimizes the likelihood of selection bias and provides reassurance that omission of trials without an established MID will not bias the result.

Individual randomized controlled trials (RCTs) often use different measurement instruments for the same construct such as disease-specific health-related quality of life (HRQL) or depression. When pooling such trials in meta-analyses, authors typically report differences between intervention and control in standard deviation (SD) units, often referred to as the standardized mean difference (SMD). This approach has statistical limitations (the same effect will appear different if population heterogeneity differs) and it is non intuitive for decision makers.

For instruments with an established minimally important difference (MID—the smallest difference patients experience as important), we have previously described the merits of reporting RCT results in relation to the MID both in individual studies [1] and in meta-analyses of RCTs using a single HRQL measure [2]. More recently, we have described an approach of reporting in MID units the pooled effects from meta-analysis of RCTs using more than one HRQL measure [3].

Reporting in MID units provides a potential solution to both the statistical and interpretational problems of reporting effects in SD units. Guidelines for interpreting MID units have been previously published [3]. The method, however, depends on a confident estimate of the MID. Anchor-based methods that examine the relationship between scores on a target instrument and some independent measure of what constitutes a small but patient-important change can provide the needed confidence, whereas the distribution-based methods that use statistical parameters associated with an instrument cannot provide the needed confidence [4], [5], [6].

These limitations in statistical approaches provide challenges for the estimation of effects in MID units in meta-analyses of results of different instruments. Those conducting systematic reviews of primary trials using multiple outcome measures will often encounter instruments for which an anchor-based MID remains unestablished.

One option to deal with this situation is to pool only the instruments that have an established anchor-based MID. This, however, will limit the power of analysis and will introduce bias if the trials that used instruments with established MIDs have underlying treatment effects that differ from those of trials using instruments for which only distribution-based MIDs are available. Dealing with this problem of power and possible selection bias requires a method of including instruments without established anchor-based MIDs. One solution to this problem is to pick a distribution-based approach that provides reasonable confidence of its relation to the MID and to use that approach for studies in which an anchor-based MID is unavailable. We explored the possibility of using such an approach.

Section snippets

Methods

Using data from a Cochrane review of chronic obstructive pulmonary disease (COPD) and a second Cochrane review of amitriptyline vs. alternative antidepressants for major depression, we evaluated the relationship between the MID (established using anchor-based methods) and the SD of the trials in each meta-analysis [7], [8]. For example, because the MID is known for the 17-item Hamilton Depression Rating Scale, we divided this MID by the SD of each of the 16 trials that used this measure. Based

Results

In the first example involving COPD, the median value for SD ratio of the SGRQ was 0.26, ranging from 0.21 to 0.34, whereas the median value for SD ratio of the CRQ was 0.51, ranging from 0.45 to 0.73. Combining both data sets, the median value for the SD ratio was 0.45, ranging from 0.21 to 0.73 (Table 1). The pooled effect estimate in MID units for the SGRQ was 0.89 (95% confidence interval [CI]: −0.15, 1.93; I2 = 0%) and for the CRQ was 1.86 (95% CI: 1.45, 2.27; I2 = 41%; Fig. 1). The pooled

Principal findings

Using two Cochrane meta-analyses, we have demonstrated an extension of our previously described approach to reporting results of continuous variables in MID units rather than SD units [3]. Hypothesizing that the MID for instruments in which an anchor-based MID remains unestablished from the ratio of MID to SD in the trials with established MIDs (the SD ratio), we have generated pooled estimates in MID units using all available data. The approach eliminates the possibility that omission of

Conclusion

Despite its limitations, the advantages of increased precision and protection against biased selection of studies recommend our new approach as an option for making results of continuous outcomes such as HRQL readily interpretable for target audiences.

References (17)

There are more references available in the full text version of this article.

Cited by (0)

View full text