ReviewA decade of individual participant data meta-analyses: A review of current practice
Introduction
The aim of individual participant data (IPD) systematic reviews and meta-analyses is to obtain all the original, raw participant data from all studies on a specified topic, in order to reanalyse the data and pool it across studies. While obtaining all the original data from all relevant studies may be time consuming and difficult, IPD meta-analysis is recognised as having many advantages over meta-analysis based on data reported in publications, and is considered the “gold standard” for meta-analysis [1], [2], [3].
Ten years ago the first review of the practice and reporting of individual participant data meta-analyses was published in Clinical Trials [4]. At that time IPD meta-analysis was still in its early stages with numbers of publications being limited before the late 1990s. The development of statistical methods in the area was also in its infancy, with key methods papers having been published only a few years before [5], [6], [7]. The review found that reporting of the processes of IPD meta-analyses was generally poor, particularly with regard to how much of the total IPD was obtained for analysis, and with poor reporting of statistical methods. Statistical analysis was also limited, with little investigation of heterogeneity, and most meta-analyses focussing on calculating overall treatment effects, with little consideration given to subgroup analyses, or how factors such as a participant's age might modify the effectiveness of a treatment. Another review performed in 2008 and published in 2010 [8] came to broadly similar conclusions.
In the ten years since that first review was published much has changed. The number of systematic reviews and meta-analyses published continues to grow, and consequently IPD meta-analysis has also grown in popularity. Advances in computing have also made meta-analyses easier to perform; most methods for IPD meta-analysis can now be implemented in all major statistical software packages. As the number of IPD meta-analyses grows there is an increased need to ensure high quality of conduct reporting of these analyses, to avoid some of the problems identified in the original review.
The Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement [9] is widely used as guidance on how to report a systematic review and meta-analysis. Recently a new PRISMA-IPD statement has been released specifically to guide the conduct and reporting of IPD reviews and meta-analyses [10]. As part of the process of creating this statement, a review of current practice in reporting of IPD reviews was conducted to identify areas where reporting was poor to inform development of the guidance. This paper presents the results of this review, updated to 2014, and expanded to consider statistical methods used in IPD meta-analyses.
Section snippets
Methods
The aim of this review was to identify published systematic reviews of medical interventions that sought to obtain and meta-analyse IPD. It was intended to obtain a representative sample of recent reviews for analysis, rather than find all such reviews, so an exhaustive database search was not performed. A MEDLINE search was performed including terms “individual participant/patient data”, “meta-analysis” and “systematic review”. This search was originally performed in January 2013 and was
Results
The original and updated search together identified 1371 potentially relevant records. After checking titles and abstracts 184 papers were considered for inclusion. After obtaining full texts (where readily available) and further checking, 100 systematic reviews of IPD were included in this review [11], [12], [13], [14], [15], [16], [17], [18], [19], [20], [21], [22], [23], [24], [25], [26], [27], [28], [29], [30], [31], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41], [42], [43],
Quality of reporting
The included reviews varied considerably in how they reported aspects of the review process, such as the search strategy, and quality assessment process. The numbers of reviews reporting key aspects of the IPD review process are summarised in Table 1. In general, aspects of the review process common to all systematic reviews were well reported, with, for example, 80% of reviews reporting details of the search strategy and search process. Inclusion and exclusion criteria were also generally
Statistical methods
The review of 2005 identified two distinct statistical approaches for IPD meta-analysis [4]. In a two-stage approach studies are analysed individually to obtain summary results for each study in the first stage, and the summary results pooled across studies as in a conventional meta-analysis in the second stage. In a one-stage approach all data from all studies are analysed simultaneously, in a single model, usually a regression model. This distinction in approaches remains in current practice.
Discussion
Much has changed in IPD systematic reviews and meta-analyses in the last ten years. The most notable change is the switch from a predominance of two-stage methods, where studies are analysed separately and summary results poled across studies; to one-stage methods where all IPD are analysed simultaneously. This is probably driven by growing familiarity with these methods, improved software, and recognition that regression models offer the greatest flexibility for IPD analysis. Another key
Conclusion
The last ten years have seen substantial changes in how IPD meta-analyses are performed, particularly with the growth of the use of one-stage regression models. There remains considerable scope for improving the quality of reporting both the process of IPD systematic reviews, and the statistical methods employed in them. It is to be hoped that the publication of the PRISMA-IPD guidelines specific to IPD reviews will improve reporting in this area [10].
Funding
The systematic review described in this paper was performed, in part, to inform the development of PRISMA-IPD. That project was supported, in part, through a National Institute for Health Research (NIHR) Senior Investigator personal award (Dr L. Stewart), and as part of the MRC–National Institute for Health Research Methodology Research Programme (grant ID 88053). The funders had no role in the design and conduct of the study; collection, management, analysis, and interpretation of the data;
Acknowledgements
We would like to thank Kath Wright for performing the literature searches for this review, and Dean Langan and Alison Smith for their assistance in the study selection and data extraction processes. We also thank the members of the PRISMA-IPD development group for their comments on early versions of this review.
References (112)
- et al.
Meta-analysis of the literature or of individual patient data: is there a difference?
Lancet
(1993) - et al.
Mohs micrographic surgery for extramammary Paget disease: a pooled analysis of individual patient data
J. Am. Acad. Dermatol.
(2013) - et al.
Aortic pulse wave velocity improves cardiovascular event prediction: an individual participant meta-analysis of prospective observational data from 17,635 subjects
J. Am. Coll. Cardiol.
(2014) - et al.
Impact of platelet reactivity on clinical outcomes after percutaneous coronary intervention. A collaborative meta-analysis of individual participant data
J. Am. Coll. Cardiol.
(2011) - et al.
Adjuvant chemotherapy after preoperative (chemo)radiotherapy and surgery for patients with rectal cancer: a systematic review and meta-analysis of individual patient data
Lancet Oncol.
(2015) - et al.
Individual patient data meta-analysis of randomized trials evaluating IL-2 monotherapy as remission maintenance therapy in acute myeloid leukemia
Blood
(2011) - et al.
The effect of age on outcomes of coronary artery bypass surgery compared with balloon angioplasty or bare-metal stent implantation among patients with multivessel coronary disease: a collaborative analysis of individual patient data from 10 randomized trials
J. Am. Coll. Cardiol.
(2012) - et al.
Long-term outcome of a routine versus selective invasive strategy in patients with non-ST-segment elevation acute coronary syndrome a meta-analysis of individual patient data
J. Am. Coll. Cardiol.
(2010) - et al.
Oral branched-chain amino acids have a beneficial effect on manifestations of hepatic encephalopathy in a systematic review with meta-analyses of randomized controlled trials
J. Nutr.
(2013) - et al.
Coronary artery bypass surgery compared with percutaneous coronary interventions for multivessel disease: a collaborative analysis of individual patient data from ten randomised trials
Lancet
(2009)
Effect of early and late compliance on the effectiveness of acamprosate in the treatment of alcohol dependence
J. Subst. Abus. Treat.
A meta-analysis of specifically designed randomized trials of sirolimus-eluting versus paclitaxel-eluting stents in diabetic patients with coronary artery disease
Am. Heart J.
Individual patient data meta-analysis of enoxaparin vs. unfractionated heparin for venous thromboembolism prevention in medical patients
J. Thromb. Haemost.
Racecadotril for childhood gastroenteritis: an individual patient data meta-analysis
Dig. Liver Dis.
A systematic review: treatment and prognosis of patients with fibrolamellar hepatocellular carcinoma
J. Am. Coll. Surg.
Attributable mortality of ventilator-associated pneumonia: a meta-analysis of individual patient data from randomised prevention studies
Lancet Infect. Dis.
Effectiveness of neuraminidase inhibitors in reducing mortality in patients admitted to hospital with influenza A H1N1pdm09 virus infection: a meta-analysis of individual participant data
Lancet Respir. Med.
Meta-analysis of comparative efficacy of increasing dose of atorvastatin versus rosuvastatin versus simvastatin on lowering levels of atherogenic lipids (from VOYAGER)
Am. J. Cardiol.
Efficacy of bupropion and the selective serotonin reuptake inhibitors in the treatment of anxiety symptoms in major depressive disorder: a meta-analysis of individual patient data from 10 double-blind, randomized clinical trials
J. Psychiatr. Res.
Platinum versus platinum-combination chemotherapy in platinum-sensitive recurrent ovarian cancer: a meta-analysis using individual patient data
Ann. Oncol.
The prognostic value of pre-operative and post-operative B-type natriuretic peptides in patients undergoing noncardiac surgery: B-type natriuretic peptide and N-terminal fragment of pro-B-type natriuretic peptide: a systematic review and individual patient data meta-analysis
J. Am. Coll. Cardiol.
Preoperative chemo(radio)therapy versus primary surgery for gastroesophageal adenocarcinoma: systematic review with meta-analysis combining individual patient and aggregate data
Eur. J. Cancer
Six versus fewer planned cycles of first-line platinum-based chemotherapy for non-small-cell lung cancer: a systematic review and meta-analysis of individual patient data
Lancet Oncol.
To IPD or not to IPD?
Eval. Health Prof.
Practical methodology of meta-analyses (overviews) using updated individual patient data
Stat. Med.
Meta-analysis of individual patient data from randomized trials; a review of methods used in practice
Cinical Trials
Meta-analysis of continuous outcome data from individual patients
Stat. Med.
A multilevel model framework for meta-analysis of clinical trials with binary outcomes
Stat. Med.
Meta-analysis of ordinal outcomes using individual patient data
Stat. Med.
Meta-analysis of individual participant data: rationale, conduct, and reporting
BMJ
Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement
BMJ
Preferred reporting items for a systematic review and meta-analysis of individual participant data: the PRISMA-IPD statement
JAMA
Aspirin in the primary and secondary prevention of vascular disease: collaborative meta-analysis of individual participant data from randomised trials
Lancet
Meta-analysis of concomitant versus sequential radiochemotherapy in locally advanced non-small-cell lung cancer
J. Clin. Oncol.
Safety of pacemaker reuse: a meta-analysis with implications for underserved nations
Circ.
Monitoring initial response to angiotensin-converting enzyme inhibitor-based regimens: an individual patient data meta-analysis from randomized, placebo-controlled trials
Hypertension
Cognitive behavioral therapy age effects in child and adolescent anxiety: an individual patient data meta-analysis
Depress. Anxiety
High-dose chemotherapy with autologous stem-cell support as adjuvant therapy in breast cancer: overview of 15 randomized trials
J. Clin. Oncol.
Infant formula supplementation with long-chain polyunsaturated fatty acids has no effect on Bayley developmental scores at 18 months of age—IPD meta-analysis of 4 large clinical trials
J. Pediatr. Gastroenterol. Nutr.
Etanercept therapy in rheumatoid arthritis and the risk of malignancies: a systematic review and individual patient data meta-analysis of randomised controlled trials
Ann. Rheum. Dis.
Effect of amifostine on survival among patients treated with radiotherapy: a meta-analysis of individual patient data
J. Clin. Oncol.
Safety and tolerability of onabotulinumtoxinA in the treatment of facial lines: a meta-analysis of individual patient data from global clinical registration studies in 1678 participants
J. Am. Acad. Dermatol.
Clinical impact of thrombectomy in acute ST-elevation myocardial infarction: an individual patient-data pooled analysis of 11 trials
Eur. Heart J.
Systematic review of purine analog treatment for chronic lymphocytic leukemia: lessons for future trials
Haematologica
Effects of smoking cessation on the outcomes of non-surgical periodontal therapy: a systematic review and individual patient data meta-analysis
J. Clin. Periodontol.
Reducing uncertainties about the effects of chemoradiotherapy for cervical cancer: a systematic review and meta-analysis of individual patient data from 18 randomized trials
J. Clin. Oncol.
Beneficial and harmful effects of anthracyclines in the treatment of childhood acute lymphoblastic leukaemia: a systematic review and meta-analysis
Br. J. Haematol.
Effects of lowering homocysteine levels with B vitamins on cardiovascular disease, cancer, and cause-specific mortality: meta-analysis of 8 randomized trials involving 37,485 individuals
Arch. Intern. Med.
Individual patient data meta-analysis of randomized evidence to assess the effectiveness of laparoscopic uterosacral nerve ablation in chronic pelvic pain
Hum. Reprod. Update
Naftidrofuryl for intermittent claudication: meta-analysis based on individual patient data
BMJ
Cited by (82)
Depression screening tool accuracy individual participant data meta-analyses: data contribution was associated with multiple factors
2023, Journal of Clinical EpidemiologyNonlinear effects and effect modification at the participant-level in IPD meta-analysis part 2: methodological guidance is available
2023, Journal of Clinical EpidemiologyNonlinear effects and effect modification at the participant-level in IPD meta-analysis part 1: analysis methods are often substandard
2023, Journal of Clinical EpidemiologyModernizing evidence synthesis for evidence-based medicine
2023, Clinical Decision Support and beyond: Progress and Opportunities in Knowledge-Enhanced Health and Healthcare