Advertisement
ADVANCED SEARCH

This Article Abstract Full Text (PDF) Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Bhandari, M. Articles by Devereaux, P.J. Search for Related Content PubMed PubMed Citation Articles by Bhandari, M. Articles by Devereaux, P.J. Related Collections Research & Publication ethics Health care funding outside of medicare Clinical research Related Article

Association between industry funding and statistically significant pro-industry findings in medical and surgical randomized trials

From the Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ont. (Bhandari, Busse, Jackowski, Montori, Schünemann, Sprague, Mears, Heels-Ansdell, Devereaux); and the Department of Surgery, University of Toronto, Toronto, Ont. (Schemitsch).

Correspondence to: Dr. Mohit Bhandari, Department of Clinical Epidemiology and Biostatistics, McMaster University Medical Centre, 1200 Main St. W, Rm. 2C3, Hamilton ON L8N 3Z5; fax 905 524-3841; bhandari{at}sympatico.ca

	Abstract

Top Abstract Methods Results Interpretation References

 
Background: Conflicting reports exist in the medical literature regarding the association between industry funding and published research findings. In this study, we examine the association between industry funding and the statistical significance of results in recently published medical and surgical trials.

Methods: We examined a consecutive series of 332 randomized trials published between January 1999 and June 2001 in 8 leading surgical journals and 5 medical journals. Each eligible study was independently reviewed for methodological quality using a 21-point index with 5 domains: randomization, outcomes, eligibility criteria, interventions and statistical issues. Our primary analysis included studies that explicitly identified the primary outcome and reported it as statistically significant. For studies that did not explicitly identify a primary outcome, we defined a "positive" study as one with at least 1 statistically significant outcome measure. We used multivariable regression analysis to determine whether there was an association between reported industry funding and trial results, while controlling for study quality and sample size.

Results: Among the 332 randomized trials, there were 158 drug trials, 87 surgical trials and 87 trials of other therapies. In 122 (37%) of the trials, authors declared industry funding. An unadjusted analysis of this sample of trials revealed that industry funding was associated with a statistically significant result in favour of the new industry product (odds ratio [OR] 1.9, 95% confidence interval [CI] 1.3–3.5). The association remained significant after adjustment for study quality and sample size (adjusted OR 1.8, 95% CI 1.1–3.0). There was a nonsignificant difference between surgical trials (OR 8.0, 95% CI 1.1–53.2) and drug trials (OR 1.6, 95% CI 1.1–2.8), both of which were likely to have a pro-industry result (relative OR 5.0, 95% CI 0.7–37.5, p = 0.14).

Interpretation: Industry-funded trials are more likely to be associated with statistically significant pro-industry findings, both in medical trials and surgical interventions.

It is not known whether these findings apply to other areas, such as surgical RCTs. Although over 70% of drug trials are funded by industry, fewer surgical trials receive such financial support.⁴ To determine whether associations between industry funding and authors' conclusions can be generalized to other specialties, we examined a consecutive series of medical and surgical trials for such associations.

	Methods

Top Abstract Methods Results Interpretation References

 
We identified RCTs published between January 1999 and June 2001 in 8 leading surgical journals (Journal of Bone and Joint Surgery [American and British volumes], Clinical Orthopaedics and Related Research, Acta Orthopaedica Scandinavica, Annals of Surgery, American Journal of Surgery, Plastic and Reconstructive Surgery and Journal of Neurosurgery) and 5 medical journals (Lancet, British Medical Journal, Journal of the American Medical Association, Annals of Internal Medicine and New England Journal of Medicine). The choice of surgical journals was based on perceived quality and impact factor revealed in a local survey of surgeons at the McMaster University Medical Centre. The choice of medical journals was also based on impact factor. We identified eligible RCTs by conducting manual and MEDLINE searches of these journals.

Each eligible study was independently reviewed for methodological quality; differences were resolved by discussion, and consensus was reached. We abstracted the following information from eligible trials: funding sources (industry-for-profit, not-for-profit, undeclared), statistical significance of outcome measures, study-quality score (using the Detsky quality index⁹), sample size, whether a priori sample-size calculations were conducted and type of intervention (drug trial, surgical trial and nonsurgical, nondrug therapy [e.g., physiotherapy trial, educational intervention]). Surgical interventions were in the fields of plastic, orthopedic, neuro- or general surgery and involved a comparison of any device, implant or technique that required a surgical procedure. A study was recorded as funded by industry if this was explicitly stated in the paper.

The Detsky quality index includes 15 items in 5 domains: randomization; outcome measures; eligibility criteria and reasons for patient exclusion (withdrawal or dropout); interventions; and statistical issues. Each domain has equal weight (4 points each). The final domain (statistical analysis) contains an extra question for trials in which findings are not significant. Thus, the maximum possible scores for statistically significant and statistically nonsignificant trials are 20 and 21, respectively. The 2 raters did not receive specific training in the use of this instrument; however, they carefully reviewed the guidelines for scoring with this index.

Our primary analysis was of studies that explicitly identified the primary outcome and reported it as statistically significant. In studies that did not explicitly identify a primary outcome, we defined a "positive" study as one with at least 1 statistically significant outcome measure. Our secondary analysis considered the number of statistically significant outcomes as a proportion of all outcomes measured in the study. In addition, we further examined whether the statistically significant outcome(s) were in favour of the industry sponsor.

We measured inter-reviewer agreement on the decision to include studies in the review and on data abstraction for 20 RCTs reviewed in duplicate. The inter-observer agreement was measured using the weighted kappa () statistic with quadratic weights. Agreement between 2 reviewers in methodological quality scores (Detsky index) was evaluated using intraclass correlation coefficients (ICCs). We compared proportions using ² analysis and means using analysis of variance. The Bonferroni method was used to correct for multiple comparisons. We conducted both unadjusted and adjusted logistic regression analyses to determine variables associated with a statistically significant study result. Initially, univariable analyses were conducted to identify factors (i.e., industry funding, study quality, sample size, type of intervention) associated with a significant study result. In an adjusted analysis, we evaluated the effect of industry funding in a multivariable logistic regression model that included sample size, study design and type of intervention.

	Results

Top Abstract Methods Results Interpretation References

 
We identified 332 RCTs: 158 drug trials, 87 surgical trials and 87 trials of other therapies. Reviewers achieved an excellent level of reliability in the identification of potential studies, data abstraction and determination of study results ( = 0.83, 0.84 and 0.88 respectively). Agreement on assessment of methodological quality was also substantial (ICC 0.79). Quality scores for surgical trials were significantly lower than those for drug trials or nonsurgical, nondrug trials (p < 0.01) (Table 1). Investigators of surgical trials were significantly less likely (p < 0.01) to report a priori sample-size calculations (24%) than investigators of either drug (61%) or nonsurgical, nondrug trials (57%).

In 122 (37%) of the 332 trials, authors declared industry funding (Table 1). Drug trials were significantly more likely than surgical or other trials to have declared industry funding (p < 0.01). Of the 122 trials declaring industry funding 48 (39%) favoured the new treatment or industry product. An unadjusted analysis of these RCTs revealed that industry funding was significantly associated with a statistically significant result (OR 1.9, 95% CI 1.3–3.5) in favour of the new industry product. After adjustment for sample size, study quality and type of intervention, those trials reporting industry funding remained significantly more likely to have a statistically significant pro-industry result (adjusted OR 1.8, 95% CI 1.1–3.0). Although the point estimate of the odds of a pro-industry conclusion in surgical trials was 5 times greater than the point estimate in drug trials (OR 8.0, 95% CI 1.1–53.2 v. OR 1.6, 95% CI 1.1–2.8 respectively), this difference was not statistically significant (relative OR 5.0, 95% CI 0.7–37.5, p = 0.14).

	Interpretation

Top Abstract Methods Results Interpretation References

 
In a review of 332 RCTs in both surgery and medicine, the authors' declaration of industry funding was significantly associated with statistically significant pro-industry results. Adjusting for variations in study quality and sample size across studies further strengthened the results. Our results fail to support the belief that variations in study quality or sample size (study power) can explain differences in trial results across industry-funded and non-industry-funded trials.

However, our findings are limited by the quality of reporting of industry funding in the journals publishing these RCTs. We reviewed each journal's "Information to Authors" section to identify policies on conflict of interest and disclosure of funding. Of 8 surgical journals, 6 required disclosure of industry funding and 2 suggested that such disclosure was appropriate. These 2 journals (Journal of Bone and Joint Surgery [British volume] and Acta Orthopaedica Scandinavica) also allowed a no-response category regarding financial disclosure. Of the 5 medical journals, 3 required disclosure of industry funding and the other 2 (JAMA and New England Journal of Medicine) suggested disclosure.

Our study has some limitations, such as our decision to use a composite scale to assess trial quality versus a component-oriented approach. Numerous checklists and scales have been reported for the evaluation of the quality of RCTs,¹⁰ and a major disadvantage of the Detsky scale, as with any such scale, is that assessments of quality depend on the information available in the published reports. As well, important aspects of study quality may not be captured by this index, such as the inappropriate use of placebos or inactive controls, or controls that are compromised by insufficient dosage or mode of administration. Composite quality scales, such as the Detsky index, may provide a useful overall assessment when comparing groups of trials, but they have been criticized, and it has been proposed that more rigorous evaluation may result if the relevant methodological aspects were identified, ideally a priori, and assessed individually.¹¹

Inferences about differences in the results of surgical and medical trials may be limited by sample size. Only 16 (18%) of all 87 identified surgical trials reported industry funding. Although the point estimate of the odds ratios suggests that surgical trials are more likely than medical trials to have pro-industry results, this difference was not statistically significant and larger studies are required to explore this finding.

Our study may be influenced by selection bias, as we elected to search only high-impact journals for trials. As well, our method of determining our primary outcome measure, although strengthened by substantial inter-observer reliability, may be questionable. Some authors² have made use of a validated scale to grade studies' conclusions regarding the extent to which an experimental intervention was favoured,¹ although others^4,6,8 have used an approach similar to ours.

Our findings are contradictory to those of some reports.^4,5,6,7 Clifford and colleagues⁴ did not find that trial outcome was associated with industry funding (p = 0.46). However, their study may have been limited by type II error and limited disclosure of funding sources. Some previous studies support our results.^1,2,3,8 In a recent meta-analysis, Bekelman and colleagues⁸ pooled 1140 original studies and found a statistically significant association between industry sponsorship and pro-industry conclusions (pooled OR 3.60, 95% CI 2.63–4.91). We pooled these results with those of Clifford and colleagues⁴ (100 trials) and our own using a random effects model. Pooling was deemed appropriate due to a nonsignificant test of heterogeneity (p > 0.1), widely overlapping confidence intervals and similarity of point estimates. Our pooled sample of 1572 trials provides a current estimate of the impact of industry funding on authors' conclusions (pooled OR 2.3, 95% CI 1.3–4.1, heterogeneity p = 0.02) (Fig. 1).

View larger version (13K): [in this window] [in a new window]   Fig. 1: Comparison of current results with a meta-analysis of 1140 medical trials and a recent study of 100 trials. Point estimates from our current sample of 158 drug trials and 87 nonsurgical, nondrug trials support previous estimates from the meta-analysis. Point estimates of the odds of pro-industry conclusions from 87 surgical trials are greater than those of drug trials and nonsurgical, nondrug trials. OR = odds ratio, CI = confidence interval, Q = quotient, df = degrees of freedom.

Our findings suggest that industry funding has a significant influence on the results of both surgical trials and drug trials. Perhaps by careful selection, industry funds trials that are most likely to reveal a benefit of the experimental intervention. Results of industry-funded trials may be influenced by inappropriate choice of comparator intervention² or by publication bias. Future exploration of the complex relation between industry-funded trials and authors' conclusions will shed further light on this issue.

ß See related article page 481

	Footnotes

 
This article has been peer reviewed.

Contributors: Mohit Bhandari drafted the manuscript and was responsible for the study concept, for critical revisions and for data acquisition, analysis and interpretation. He provided statistical expertise, administrative, technical and material support and study supervision. Emil Schemitsch was responsible for study concept and design, data analysis and interpretation, and critical revisions. Jason Busse and Diane Jackowski were responsible for data acquisition and critical revisions. Derek Mears was responsible for data acquisition. P.J. Devereaux was responsible for data analysis and interpretation, for critical revisions and for providing statistical expertise. Victor Montori and Holger Schünemann were responsible for data analysis and interpretation and for critical revisions. Sheila Sprague was responsible for critical revisions and provided administrative, technical and material support. Diane Heels-Ansdell provided statistical expertise, analysis and interpretation of data and critical revision of the manuscript. All authors gave approval of the final version to be published.

Acknowledgements: Mohit Bhandari is funded, in part, by a Clinical Scientist Fellowship, Department of Clinical Epidemiology and Biostatistics, McMaster University. Jason Busse is funded by a Canadian Institutes of Health Research Fellowship Award. P.J. Devereaux is funded by a Heart and Stroke Foundation of Canada/Canadian Institutes of Health Research Fellowship Award.

Competing interests: None declared.

	References

Top Abstract Methods Results Interpretation References

 

1. Kjaergard LL, Als-Nielsen B. Association between competing interests and authors' conclusions: epidemiological study of randomised clinical trials published in the BMJ. BMJ 2002;325(7358):249-53. [Abstract/Free Full Text]
2. Djulbegovic B, Lacevic M, Cantor A, Fields KK, Bennett CL, Adams JR, et al. The uncertainty principle and industry-sponsored research. Lancet 2000; 356(9230):635-8.[Medline]
3. Lexchin J, Bero LA, Djulbegovic B, Clark O. Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 2003; 326(7400):1167-70.[Abstract/Free Full Text]
4. Clifford T, Barrowman N, Moher D. Funding source, trial outcome and reporting quality: Are they related? Results of a pilot study. BMC Health Serv Res 2002;2(1):18-24.[Medline]
5. Davidson RA. Source of funding and outcome of clinical trials. J Gen Intern Med 1986;1(3):155-8.[Medline]
6. Rochon PA, Gurwitz JH, Simms RW, Fortin P, Felson DT, Minaker KL, et al. A study of manufacturer-supported trials of nonsteroidal anti-inflammatory drugs in the treatment of arthritis. Arch Intern Med 1994;154(2):157-63.[Abstract/Free Full Text]
7. Cho MK, Bero LA. The quality of drug studies published in symposium proceedings. Ann Intern Med 1996;124(5):485-9.[Abstract/Free Full Text]
8. Bekelman JE, Li Y, Gross CP. Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA 2003;289(4):454-65.[Abstract/Free Full Text]
9. Detsky AS, Naylor CD, O'Rourke K, McGeer AJ, L'Abbe K. Incorporating variations in the quality of individual randomized trials into meta-analysis. J Clin Epidemiol 1992;45(3):225-65.
10. Moher D, Jadad AR, Nichol G, Penman M, Tugwell P, Walsh S. Assessing the quality of randomized controlled trials: an annotated bibliography of scales and checklists. Control Clin Trials 1995;16(1):62-73.[Medline]
11. Juni P, Witschi A, Bloch R, Egger M. The hazards of scoring the quality of clinical trials for meta-analysis. JAMA 1999;282(11):1054-60.[Abstract/Free Full Text]

This article has been cited by other articles:

				K. Okike, M. S. Kocher, E. X. Wei, C. T. Mehlman, and M. Bhandari Accuracy of Conflict-of-Interest Disclosures Reported by Physicians N. Engl. J. Med., October 8, 2009; 361(15): 1466 - 1474. [Abstract] [Full Text] [PDF]

				S. Greenland Accounting for uncertainty about investigator bias: disclosure is informative: How could disclosure of interests work better in medicine, epidemiology and public health? J Epidemiol Community Health, August 1, 2009; 63(8): 593 - 598. [Full Text] [PDF]

				R. S. Irwin The Role of Conflict of Interest in Reporting of Scientific Information Chest, July 1, 2009; 136(1): 253 - 259. [Abstract] [Full Text] [PDF]

				C. M. Booth, D. W. Cescon, L. Wang, I. F. Tannock, and M. K. Krzyzanowska Evolution of the Randomized Controlled Trial in Oncology Over Three Decades J. Clin. Oncol., November 20, 2008; 26(33): 5458 - 5464. [Abstract] [Full Text] [PDF]

				G. H. Guyatt, D. J. Cook, R. Jaeschke, S. G. Pauker, and H. J. Schunemann Grades of Recommendation for Antithrombotic Agents: American College of Chest Physicians Evidence-Based Clinical Practice Guidelines (8th Edition) Chest, June 1, 2008; 133(6_suppl): 123S - 131S. [Abstract] [Full Text] [PDF]

				J Avouac, J Wipff, A Kahan, and Y Allanore Effects of oral treatments on exercise capacity in systemic sclerosis related pulmonary arterial hypertension: a meta-analysis of randomised controlled trials Ann Rheum Dis, June 1, 2008; 67(6): 808 - 814. [Abstract] [Full Text] [PDF]

				B. Reider Passing the Headline Test Am. J. Sports Med., November 1, 2007; 35(11): 1807 - 1808. [Full Text] [PDF]

				R. P. Riechelmann, L. Wang, A. O'Carroll, and M. K. Krzyzanowska Disclosure of Conflicts of Interest by Authors of Clinical Trials and Editorials in Oncology J. Clin. Oncol., October 10, 2007; 25(29): 4642 - 4647. [Abstract] [Full Text] [PDF]

				M. Gogol Standards in dementia care Age Ageing, September 1, 2007; 36(5): 599 - 600. [Full Text] [PDF]

				J. R. Lynch, M. R.A. Cunningham, W. J. Warme, D. C. Schaad, F. M. Wolf, and S. S. Leopold Commercially Funded and United States-Based Research Is More Likely to Be Published; Good-Quality Studies with Negative Outcomes Are Not J. Bone Joint Surg. Am., May 1, 2007; 89(5): 1010 - 1018. [Abstract] [Full Text] [PDF]

				K. Okike and M. S. Kocher The Legal and Ethical Issues Surrounding Financial Conflict of Interest in Orthopaedic Research J. Bone Joint Surg. Am., April 1, 2007; 89(4): 910 - 913. [Full Text] [PDF]

				R. S. Irwin Clinical Trial Registration Promotes Patient Protection and Benefit, Advances the Trust of Everyone, and Is Required Chest, March 1, 2007; 131(3): 639 - 641. [Full Text] [PDF]

				K. Okike, M. S. Kocher, C. T. Mehlman, and M. Bhandari Conflict of Interest in Orthopaedic Research. An Association Between Findings and Funding in Scientific Presentations J. Bone Joint Surg. Am., March 1, 2007; 89(3): 608 - 613. [Abstract] [Full Text] [PDF]

				M. Glick Opprobrium or opportunity? J Am Dent Assoc, November 1, 2006; 137(11): 1484 - 1486. [Full Text] [PDF]

				D. K. Benjamin Jr, P. B. Smith, M. D. Murphy, R. Roberts, L. Mathis, D. Avant, R. M. Califf, and J. S. Li Peer-reviewed publication of clinical trials completed for pediatric exclusivity. JAMA, September 13, 2006; 296(10): 1266 - 1273. [Abstract] [Full Text] [PDF]

				J. S. Ross and H. M. Krumholz Abstract Acceptance and Blinded Peer Review--Reply JAMA, September 13, 2006; 296(10): 1230 - 1231. [Full Text] [PDF]

				V. M. Montori, Y. G. Wang, P. Alonso-Coello, and S. Bhagra Systematic Evaluation of the Quality of Randomized Controlled Trials in Diabetes Diabetes Care, August 1, 2006; 29(8): 1833 - 1838. [Abstract] [Full Text] [PDF]

				I. Chalmers From optimism to disillusion about commitment to transparency in the medico-industrial complex J R Soc Med, July 1, 2006; 99(7): 337 - 341. [Full Text] [PDF]

				J. J. Jacobs, J. O. Galante, S. K. Mirza, and T. Zdeblick Symposium. Relationships with Industry: Critical for New Technology or an Unnecessary Evil? J. Bone Joint Surg. Am., July 1, 2006; 88(7): 1650 - 1663. [Full Text] [PDF]

				R E Ferner and S. E McDowell How NICE may be outflanked. BMJ, May 27, 2006; 332(7552): 1268 - 1271. [Full Text] [PDF]

				P. M Ridker and J. Torres Reported outcomes in major cardiovascular clinical trials funded by for-profit and not-for-profit organizations: 2000-2005. JAMA, May 17, 2006; 295(19): 2270 - 2274. [Abstract] [Full Text] [PDF]

				D. M. Arnold, D. A. Fergusson, A. K.C. Chan, R. J. Cook, G. A. Fraser, W. Lim, M. A. Blajchman, and D. J. Cook Avoiding transfusions in children undergoing cardiac surgery: a meta-analysis of randomized trials of aprotinin. Anesth. Analg., March 1, 2006; 102(3): 731 - 737. [Abstract] [Full Text] [PDF]

				N. Siegfried, M. Clarke, and J. Volmink Randomised controlled trials in Africa of HIV and AIDS: descriptive study and spatial distribution BMJ, October 1, 2005; 331(7519): 742. [Abstract] [Full Text] [PDF]

				D. C. Goodman When an Asthma Drug Has an Inferiority Complex: A Noninferiority Trial Pediatrics, August 1, 2005; 116(2): 493 - 495. [Full Text] [PDF]

				P. J. Sterk and K. F. Rabe Serving researchers, the impact factor and other conflicts of interest. Eur. Respir. J., January 1, 2005; 25(1): 3 - 5. [Full Text] [PDF]

				J. R Lexchin Implications of Pharmaceutical Industry Funding on Clinical Research Ann. Pharmacother., January 1, 2005; 39(1): 194 - 197. [Abstract] [Full Text] [PDF]

				R. M. Califf, S. E. Nissen, A. N. DeMaria, E. M. Ohman, B. Pitt, J. T. Willerson, D. W. Bilheimer, J. N. Cohn, D. W. Feigal Jr, L. Hampson, et al. Task force 2: Investigator participation in clinical research J. Am. Coll. Cardiol., October 19, 2004; 44(8): 1729 - 1736. [Full Text] [PDF]

				R. M. Califf, S. E. Nissen, A. N. DeMaria, E. M. Ohman, B. Pitt, J. T. Willerson, D. W. Bilheimer, J. N. Cohn, D. W. Feigal Jr, L. Hampson, et al. Task Force 2: Investigator Participation in Clinical Research Circulation, October 19, 2004; 110(16): 2517 - 2524. [Full Text] [PDF]

				D. Rennie Trial Registration: A Great Idea Switches From Ignored to Irresistible JAMA, September 15, 2004; 292(11): 1359 - 1362. [Full Text] [PDF]

				J A C Delaney Tracking system for studies should be in place BMJ, July 17, 2004; 329(7458): 173 - 173. [Full Text]

				M. A. Voysey SSRI treatment for under-18s Can. Med. Assoc. J., June 8, 2004; 170(12): 1771 - 1771. [Full Text] [PDF]

				M. Bhandari, V. M. Montori, and P.J. Devereaux SSRI treatment for under-18s Can. Med. Assoc. J., June 8, 2004; 170(12): 1771 - 1771. [Full Text] [PDF]

				L. Hirsch Randomized clinical trials: What gets published, and when? Can. Med. Assoc. J., February 17, 2004; 170(4): 481 - 483. [Full Text] [PDF]

This Article Abstract Full Text (PDF) Submit a response Alert me when this article is cited Alert me when eLetters are posted Alert me if a correction is posted Citation Map Services Email this article to a friend Similar articles in this journal Similar articles in PubMed Alert me to new issues of the journal Download to citation manager Citing Articles Citing Articles via HighWire Citing Articles via Google Scholar Google Scholar Articles by Bhandari, M. Articles by Devereaux, P.J. Search for Related Content PubMed PubMed Citation Articles by Bhandari, M. Articles by Devereaux, P.J. Related Collections Research & Publication ethics Health care funding outside of medicare Clinical research Related Article

HOME	CURRENT ISSUE	PAST ISSUES	COLLECTIONS	HELP	SEARCH
Copyright 1995-2004, Canadian Medical Association. All rights reserved. ISSN 1488-2329 (e) 0820-3946 (p) All editorial matter in CMAJ represents the opinions of the authors and not necessarily those of the Canadian Medical Association.