- © 2004 Canadian Medical Association or its licensors
In this issue (page 477) Bhandari and colleagues1 report an association between industry funding of clinical trials and findings of statistical significance. The authors extend previous studies of drug trials2,3,4,5,6,7 by including surgical and nonsurgical, nondrug trials and suggest that industry funding has a significant influence on the results of both surgical and drug trials. To many readers, the findings may seem self-evident. Hardly a week passes without an article, commentary or editorial in a prominent medical journal raising concerns about conflicts of interest, particularly financial, and concluding that research sponsored by industry is “biased.” I do not believe that use of the term “industry” is any more appropriate than “all physicians” when discussing standards of medical practice. Pharmaceutical companies differ in their approaches to clinical trials and their publication; here I speak from the perspective of Merck & Co., Inc. (Merck-Frosst in Canada; MSD outside North America).
Several studies have shown associations between the funding of clinical trials by for-profit organizations and the reporting of positive results2,3,4,5,6 or conclusions;8,9 others have not.7,10 Broadly, explanations offered for this include the following: companies fund only studies that will demonstrate positive outcomes for their product(s); studies are designed or conducted poorly (including inappropriate comparators); and negative or unfavourable studies are not published.4 I will address each of these issues in turn.
First, during the product development and registration process, pharmaceutical companies often need to conduct some placebo-controlled trials as a requirement for regulatory approval; these are more likely than other types of trials (e.g., active comparators) to be “positive.” Furthermore, since companies can undertake only a finite number of studies concerning a product's safety or efficacy profile, studies thought more likely to succeed receive higher priority. This is not unlike the situation of researchers in academia and government, who have views that influence the research questions they ask and the studies they conduct. Perhaps, based on their experience with preclinical and early-phase clinical studies of a candidate product, companies are able to select trials in later stages of development with a higher likelihood of positive return, but researchers in all of these domains simply cannot predict the results of all trials in advance.5,11,12,13,14
Second, numerous studies have shown that the operational quality of trials sponsored by pharmaceutical companies is as good as, if not better than, that of trials funded by other sources.4,5,6,7,9,10,15 Protocols for trials sponsored by Merck undergo extensive clinical and statistical reviews internally as well as by study investigators, regulatory agencies, institutional review boards and ethical review committees. Except in pilot studies, end-point measures and methods of analysis in all hypothesis-testing studies are prespecified. Quality-assurance and quality-control procedures (including auditing source documents) verify the integrity of the data collected. It is true that, for active comparison trials, selection of inappropriate comparators (including dosing) can influence study outcomes.3,16 When such trials are conducted by Merck, the manufacturers' precise dosing and administration instructions are followed for all products.12,17,18 These processes minimize bias and strengthen the credibility of the study outcomes, enhancing the likelihood that the study will be received favourably by peer reviewers for publication and in the regulatory review process. A positive response from regulatory reviewers will permit the sponsor to promote the study (with fair balance) to prescribers; promotional efforts have been shown in Canada to accelerate the adoption of important new evidence from randomized controlled trials (RCTs).19
Third, publication bias — the so-called “file-drawer” phenomenon, in which trials with negative or inconclusive results are not submitted for publication (or accepted by medical journals) — has been reported.20,21,22,23 Industry funding and financial conflict of interest are often cited in this regard, and there have been some well-known instances of delay or suppression of publication of unfavourable results,24 but this should not be generalized to all companies. Publication bias (or lag) has been documented in research regardless of sponsorship,20,21,22,23,25 and a recent analysis showed that phase III oncology trials sponsored by pharmaceutical companies were published significantly sooner than were trials sponsored by cooperative groups (consortia of investigators largely funded by the National Institutes of Health) or those whose sponsorship was not specified.26 Recently, Merck has adopted guidelines in which we commit to publish the results of hypothesis-testing clinical trials regardless of outcome.27 We recognize the importance of results from adequately powered negative trials (as opposed to trials with type II errors) and the ethical obligation to publish such data.12,18,28
This is not our only obligation, however. In Western societies, medical product development is left to the marketplace, not to academia or government agencies.29 For drugs and vaccines, this is the pharmaceutical industry, which, in 2002, spent more than US$32 billion on research and development.30 As a public corporation whose mission is to discover and develop innovative products that meet unmet medical needs, Merck (like other companies) is obliged to protect proprietary information and intellectual property, including aspects of the design of clinical trials of investigational agents and the very existence of certain studies. Exploratory or pilot studies (usually observational or early-phase trials) enable decision-making at critical development milestones and better design of subsequent large-scale trials to rigorously test hypotheses for both product safety and efficacy. Premature disclosure of proprietary information by Merck (or other companies) can result in significant competitive disadvantage and loss of incentive or reward for new product development. Hence we, like others, do not concur with calls for mandatory registration of all clinical trials at their inception to redress publication bias;31,32,33 rather, we commit to publish trials as noted above.
Bhandari and colleagues retrieved 332 reports of RCTs — 158 drug trials, 87 surgical trials and 87 nonsurgical, nondrug trials — published in 8 surgical and 5 medical journals. Industry funding was declared in 37% of the reports, and these trials were more likely to be associated with statistically significant results favouring the sponsor's product (adjusted odds ratio 1.8, 95% confidence interval 1.1–3.0) than were trials funded by government/foundations or those without a funding source declared. Surgical trials were more likely than drug trials to be associated with a “pro-industry” result, but the difference was not significant.
Studies that attempt to examine sources of bias in the design, analysis, reporting and publication of RCTs are difficult to do. Usually, only published trials are examined, and these are limited by the data reported. In the current analysis, 44% of the studies had no source of funding reported, which was the primary measure of interest.1 By analogy, CMAJ readers would be sceptical of any RCT that lacked primary outcome data for nearly half of the enrolled patients. In addition, trials with different designs and sample sizes were combined. Although the authors adjusted their analysis for sample size, study design and type of intervention, it is not clear what proportion of trials were placebo-controlled versus active comparator studies, or trials of alternative therapies.
Other aspects of the study make the results difficult to interpret. The primary analysis was of trials that explicitly identified a primary outcome and reported it as statistically significant, but studies that did not identify a primary outcome measure were included as positive if there was “at least 1 statistically significant outcome measure” — a very loose criterion. Alternatively, 2 interventions may not differ statistically in efficacy, but the newer product may be shown to have a better safety profile or tolerability;17 such studies would certainly be considered “positive” by the sponsor, but it is unclear how these were classified. Bhandari and colleagues give the median number of authors affiliated with industry, but the importance of this figure is unclear without knowing the total number of authors (lack of denominator). They also do not report how they calculated the odds ratios for the association between industry funding and significant results in favour of the new industry product: specifically, how were studies coded in which funding was not declared? Were they combined with studies funded by government/foundations? It would have been clearer if the actual proportions of trials favouring the industry product had been presented for each category of study (not just those funded by industry), along with the derived odds ratios.
The data show that 39% of the 122 industry-funded trials had outcomes favouring the sponsor's product. The rate for the 98 industry-sponsored drug trials was 34%. Thus, in two-thirds (66%) of the industry-sponsored drug trials, the results did not favour the sponsor's product. Of all the 158 drug trials (regardless of funding source), 74% showed a statistically significant outcome, more than twice the rate for the subset of industry-funded drug trials. These data do not support the interpretation that results of industry-funded studies (at least drug trials) are biased toward positive outcomes; perhaps the results in the small subset of surgical trials heavily influenced the odds ratios calculated by the authors.
In summary, Bhandari and colleagues report that industry funding was associated with increased odds of finding statistically significant results in RCTs of drugs and surgical interventions. The relation is of modest degree, and questions remain about both the data and their interpretation. Additional studies are needed to clarify the relation between funding of clinical research, research outcomes and trial publication — but we may still not know the “final answer.”
𝛃 See related article page 477
Footnotes
-
Competing interests: Dr. Hirsch is employed by Merck & Co., Inc. (Merck Research Laboratories).