Advertisement
ADVANCED SEARCH

This Article Figures Only Full Text Full Text (PDF) Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Ioannidis, J. P.A. Articles by Trikalinos, T. A. Search for Related Content PubMed PubMed Citation Articles by Ioannidis, J. P.A. Articles by Trikalinos, T. A. Related Collections Statistics & Research Methods

The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey

From the Tufts University School of Medicine (Ioannidis, Trikalinos), Boston, Mass., and the Clinical Trials and Evidence-Based Medicine Unit and Clinical and Molecular Epidemiology Unit, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, and the Biomedical Research Institute, Foundation for Research and Technology — Hellas (Ioannidis), Ioannina, Greece.

Correspondence to: Dr. John Ioannidis, Department of Hygiene and Epidemiology, University of Ioannina School of Medicine, 45 110 Ioannina, Greece; fax +30 26510 97867; jioannid{at}cc.uoi.gr

Background: Statistical tests for funnel-plot asymmetry are common in meta-analyses. Inappropriate application can generate misleading inferences about publication bias. We aimed to measure, in a survey of meta-analyses, how frequently the application of these tests would be not meaningful or inappropriate.

Methods: We evaluated all meta-analyses of binary outcomes with é 3 studies in the Cochrane Database of Systematic Reviews (2003, issue 2). A separate, restricted analysis was confined to the largest meta-analysis in each of the review articles. In each meta-analysis, we assessed whether criteria to apply asymmetry tests were met: no significant heterogeneity, I2 < 50%, é 10 studies (with statistically significant results in at least 1) and ratio of the maximal to minimal variance across studies > 4. We performed a correlation and 2 regression asymmetry tests and evaluated their concordance. Finally, we sampled 60 meta-analyses from print journals in 2005 that cited use of the standard regression test.

Results: A total of 366 of 6873 (5%) and 98 of 846 meta-analyses (12%) in the wider and restricted Cochrane data set, respectively, would have qualified for use of asymmetry tests. Asymmetry test results were significant in 7%–18% of the meta-analyses. Concordance between the 3 tests was modest (estimated k 0.33–0.66). Of the 60 journal meta-analyses, 7 (12%) would qualify for asymmetry tests; all 11 claims for identification of publication bias were made in the face of large and significant heterogeneity.

Interpretation: Statistical conditions for employing asymmetry tests for publication bias are absent from most meta-analyses; yet, in medical journals these tests are performed often and interpreted erroneously.

This article has been cited by other articles:

				D. L Schriger, D. G Altman, J. A Vetter, T. Heafner, and D. Moher Forest plots in reports of systematic reviews: a cross-sectional study reviewing current practice Int. J. Epidemiol., January 28, 2010; (2010): dyp370v2 - dyp370. [Abstract] [Full Text] [PDF]

				A. Liberati, D. G Altman, J. Tetzlaff, C. Mulrow, P. C Gotzsche, J. P A Ioannidis, M. Clarke, P J Devereaux, J. Kleijnen, and D. Moher The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration BMJ, December 4, 2009; 339(jul21_1): b2700 - b2700. [Abstract] [Full Text] [PDF]

				M. Srinivasan, E. Taioli, and C. C.Ragin Human papillomavirus type 16 and 18 in primary lung cancers--a meta-analysis Carcinogenesis, October 1, 2009; 30(10): 1722 - 1728. [Abstract] [Full Text] [PDF]

				A. Liberati, D. G. Altman, J. Tetzlaff, C. Mulrow, P. C. Gotzsche, J. P.A. Ioannidis, M. Clarke, P. J. Devereaux, J. Kleijnen, and D. Moher The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration Ann Intern Med, August 18, 2009; 151(4): W-65 - W-94. [Abstract] [Full Text] [PDF]

				S. G Moreno, A. J Sutton, E. H Turner, K. R Abrams, N. J Cooper, T. M Palmer, and A E Ades Novel methods to deal with publication biases: secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications BMJ, August 7, 2009; 339(aug07_1): b2981 - b2981. [Abstract] [Full Text] [PDF]

				A. Spelsberg, A. Martiny, and P S Schoenhoefer Is disclosure of potential conflicts of interest in medicine and public health sufficient to increase transparency and decrease corruption?: How could disclosure of interests work better in medicine, epidemiology and public health? J Epidemiol Community Health, August 1, 2009; 63(8): 603 - 605. [Full Text] [PDF]

				I. J. Lean, A. R. Rabiee, T. F. Duffield, and I. R. Dohoo Invited review: Use of meta-analysis in animal health and reproduction: Methods and applications J Dairy Sci, August 1, 2009; 92(8): 3545 - 3565. [Abstract] [Full Text] [PDF]

				M. Crowther, M. A. Vickers, and A. Avenell A case of apples and pears? Blood, June 11, 2009; 113(24): 6259 - 6260. [Full Text] [PDF]

				P. S. Shah, A. Ohlsson, and on behalf of the Knowledge Synthesis Group on Dete Effects of prenatal multimicronutrient supplementation on pregnancy outcomes: a meta-analysis Can. Med. Assoc. J., June 9, 2009; 180(12): E99 - E108. [Abstract] [Full Text] [PDF]

				D. E.G. Griesdale, R. J. de Souza, R. M. van Dam, D. K. Heyland, D. J. Cook, A. Malhotra, R. Dhaliwal, W. R. Henderson, D. R. Chittock, S. Finfer, et al. Intensive insulin therapy and mortality among critically ill patients: a meta-analysis including NICE-SUGAR study data Can. Med. Assoc. J., April 14, 2009; 180(8): 821 - 827. [Abstract] [Full Text] [PDF]

				C. Barbui, A. Signoretti, S. Mule, M. Boso, and A. Cipriani Does the Addition of a Second Antipsychotic Drug Improve Clozapine Treatment? Schizophr Bull, March 1, 2009; 35(2): 458 - 468. [Abstract] [Full Text] [PDF]

				S. Boccia, M. Hashibe, P. Galli, E. De Feo, T. Asakage, T. Hashimoto, A. Hiraki, T. Katoh, T. Nomura, A. Yokoyama, et al. Aldehyde Dehydrogenase 2 and Head and Neck Cancer: A Meta-analysis Implementing a Mendelian Randomization Approach Cancer Epidemiol. Biomarkers Prev., January 1, 2009; 18(1): 248 - 254. [Abstract] [Full Text] [PDF]

				R. Gonzalez, J. Zamora, J. Gomez-Camarero, L.-M. Molinero, R. Banares, and A. Albillos Meta-analysis: Combination Endoscopic and Drug Therapy to Prevent Variceal Rebleeding in Cirrhosis Ann Intern Med, July 15, 2008; 149(2): 109 - 122. [Abstract] [Full Text] [PDF]

				G. ter Riet MD PhD, J. S. Cnossen MD, J. A.M. van der Post MD PhD, B. W. Mol MD PhD, R. K. Morris MD, and K. S. Khan MD Use of Doppler ultrasonography to predict pre-eclampsia Can. Med. Assoc. J., July 1, 2008; 179(1): 53 - 54. [Full Text] [PDF]

				D. E. Karageorgopoulos MD, K. P. Giannopoulou MD, A. P. Grammatikos MD MSc, G. Dimopoulos MD, and M. E. Falagas MD DSc Fluoroquinolones compared with {beta}-lactam antibiotics for the treatment of acute bacterial sinusitis: a meta-analysis of randomized controlled trials Can. Med. Assoc. J., March 25, 2008; 178(7): 845 - 854. [Abstract] [Full Text] [PDF]

				S. Boccia, R. Hung, G. Ricciardi, F. Gianfagna, M. P. A. Ebert, J.-Y. Fang, C.-M. Gao, T. Gotze, F. Graziano, M. Lacasana-Navarro, et al. Meta- and Pooled Analyses of the Methylenetetrahydrofolate Reductase C677T and A1298C Polymorphisms and Gastric Cancer Risk: A Huge-GSEC Review Am. J. Epidemiol., March 1, 2008; 167(5): 505 - 516. [Abstract] [Full Text] [PDF]

				E. H. Turner, A. M. Matthews, E. Linardatos, R. A. Tell, and R. Rosenthal Selective Publication of Antidepressant Trials and Its Influence on Apparent Efficacy N. Engl. J. Med., January 17, 2008; 358(3): 252 - 260. [Abstract] [Full Text] [PDF]

				E. Ioannidou, I. I. Siempos, and M. E. Falagas Administration of antimicrobials via the respiratory tract for the treatment of patients with nosocomial pneumonia: a meta-analysis J. Antimicrob. Chemother., December 1, 2007; 60(6): 1216 - 1226. [Abstract] [Full Text] [PDF]

				M. H. Pittler MD PhD, E. M. Brown BSc, and E. Ernst MD PhD Static magnets for reducing pain: systematic review and meta-analysis of randomized trials Can. Med. Assoc. J., September 25, 2007; 177(7): 736 - 742. [Abstract] [Full Text] [PDF]

				R. Zarychanski MD, A. F. Turgeon MD MSc, L. McIntyre MD MHSc, and D. A. Fergusson MHA PhD Erythropoietin-receptor agonists in critically ill patients: a meta-analysis of randomized controlled trials Can. Med. Assoc. J., September 25, 2007; 177(7): 725 - 734. [Abstract] [Full Text] [PDF]

				J. P. Ioannidis and T. A Trikalinos Authors' response to V Johnson and Y Yuan Clinical Trials, June 1, 2007; 4(3): 256 - 257. [PDF]

HOME	CURRENT ISSUE	PAST ISSUES	COLLECTIONS	HELP	SEARCH
Copyright 1995-2007, Canadian Medical Association. All rights reserved. ISSN 1488-2329 (e) 0820-3946 (p) All editorial matter in CMAJ represents the opinions of the authors and not necessarily those of the Canadian Medical Association.