Availability of large-scale evidence on specific harms from systematic reviews of randomized trials
Section snippets
Methods
We selected systematic reviews of randomized controlled trials for which there were quantitative data on the occurrence of at least one very specific adverse event in each study and per study arm for ≥4000 subjects, and for which a formal meta-analysis of these data had been performed. The cutoff of 4000 subjects was decided a priori. With 4000 subjects assigned randomly and approximately equally to two intervention arms, there is about 80% power to detect an adverse event due to an
Results
Of 1754 systematic reviews, 27 had been withdrawn and 1589 did not have at least 4000 randomized subjects. There were 138 systematic reviews with a sample size of ≥4000 for at least one quantitative comparison (Table 1). Of those, only 25 (18%) provided eligible data on specific harms.
Among reviews with at least 4000 subjects but no eligible data on harms, most did not provide separate quantitative data on any adverse events (specific or not), but about a third provided some information on
Discussion
Systematic reviews of randomized controlled trials rarely provide large-scale evidence on specific, well-defined adverse events associated with the tested interventions. Only 25 systematic reviews in the entire Cochrane Database of Systematic Reviews contained such evidence on at least one type of harm. More than three quarters of systematic reviews with at least 4000 subjects in randomized trials lacked such information. The Cochrane Library is known for the high quality of its reviews (5, 6)
Acknowledgment
We are thankful to Professor Jan P. Vandenbroucke, Department of Clinical Epidemiology, University of Leiden Medical School, Leiden, The Netherlands, for discussing our study protocol and for critical reading and suggestions for the discussion in the manuscript.
References (17)
Methodology and overt and hidden bias in reports of 196 double-blind trials of nonsteroidal, antiinflammatory drugs in rheumatoid arthritis [published correction in Control Clin Trials. 1989;10:356]
Control Clin Trials
(1989)- et al.
Reporting of adverse effects in clinical trials should be improved: lessons from acute postoperative pain
J Pain Symptom Manage
(1999) - et al.
Risk of hospital admission for idiopathic venous thromboembolism among users of postmenopausal oestrogens
Lancet
(1996) - et al.
Improving safety reporting from randomised trials
Drug Saf
(2002) - et al.
Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas
JAMA
(2001) The discovery of drug-induced illness
N Engl J Med
(1977)- et al.
The role of meta-analysis in the regulatory process for foods, drugs, and devices
JAMA
(1999) - et al.
Systematic reviews and meta-analyses on treatment of asthma: critical evaluation
BMJ
(2000)
Cited by (60)
Most healthcare interventions tested in Cochrane Reviews are not effective according to high quality evidence: a systematic review and meta-analysis
2022, Journal of Clinical EpidemiologyComparative rates of harms in randomized trials from more developed versus less developed countries may be different
2016, Journal of Clinical EpidemiologyAdverse Effects of Psychotropic Medications: A Call to Action
2016, Psychiatric Clinics of North AmericaCitation Excerpt :Sometimes investigators or sponsors may change how adverse events are assessed or reported (either the definition of the adverse event or the method of assessment) from commonly used approaches. This may alter the conclusions, eg, the conclusion using an idiosyncratic way of assessing or reporting adverse effects may be that one medication has a better adverse event profile than another.17 There are 4 general approaches to identify adverse effects, but each of them has significant limitations.
Basic study design influences the results of orthodontic clinical investigations
2015, Journal of Clinical EpidemiologyCitation Excerpt :Estimations of a treatment's adverse effects may be prone to different biases than its efficacy [32]. RCTs may not be large enough or may not have sufficiently large follow-up to identify some long-term harms [33,41–44]. Moreover, generalizability of the RCTs' results may be limited for various reasons [45]; for example, as high-risk patients are often excluded from trials [32,33,46].
Side effects are incompletely reported among systematic reviews in gastroenterology
2015, Journal of Clinical EpidemiologyCitation Excerpt :There is no standardized validated tool to assess harms reporting in systematic reviews. We pilot tested a number of previously published instruments [1–3,5,20,21] and chose and adapted an extension of the Consolidated Standards of Reporting Trials statement developed specifically for harms reporting [17] as this checklist captures many aspects (such as clearly defining adverse events, providing numerical data on incidence, and recording methods of data collection [20,22]) (Table 1). One researcher (S.E.M.) used a standardized spreadsheet to extract study characteristics as well as quantitative and qualitative parameters of harms reporting.