Clinical trial registries becoming a reality, but long-term effects remain uncertain

Advocates for clinical trial registration have in recent years gained several formidable allies, including editors of high-profile medical journals and the United States government, but some warn that publishing incentives and mandatory-registration laws, while strides in the right direction, will not guarantee change in a culture that has traditionally been as transparent as chocolate milk.

“The notion of endorsement is great. The notion of adherence is more important,” says David Moher, director of the University of Ottawa Evidence-based Practice Center in Ottawa, Ontario.

As early as 1993, Moher, senior scientist in clinical epidemiology with the Ottawa Health Research Institute, warned that the failure to require scientific investigators to register trials had “led to publication bias, research ethics boards’ approval of protocols without assurance that results will be written up and made available, and granting agencies’ inability to determine the direction of the research being conducted” (CMAJ 1993;149[11]:1657–8).

Much has changed, however, since the early 1990s. In 1997, the US National Institutes of Health launched ClinicalTrials.gov, now the world’s largest clinical trial registry. In 2004, then-New York attorney general Eliot Spitzer made headlines when he filed a lawsuit against GlaxoSmithKline, accusing the drug giant of hiding study data that indicated its best-selling antidepressant paroxetine (Paxil) might increase suicidal tendencies in children. A year later, the International Committee of Medical Journal Editors announced that many top medical journals would no longer publish unregistered studies. Within a month, the number of registrations in ClinicalTrials.gov jumped from 13 153 to 22 714.

In May 2007, another registration milestone was reached when the World Health Organization formally launched its International Clinical Trial Registry Platform and designated “Primary Registries” in China, the United Kingdom, the US and Australia. The most significant change, however, occurred in December 2007. That’s when Section 801 of the US Food and Drug Administration (FDA) Amendment Act, which made clinical trial registration mandatory (except for phase I drug trials), came into effect. Those who break this law could lose grant money and be penalized up to US$10 000 a day.

In Canada, there are no national regulations that require researchers to register trials, though Health Canada is exploring that option. In the meantime, Health Canada says that it encourages trial sponsors to register trials in existing registers.

But since 2004, the Canadian Institutes of Health Research (CIHR) has required that clinical trials it funds be registered in the UK-based Current Controlled Trials International Standard Randomised Controlled Trials Number Register. In 2008/09, some 91 trials were obliged to register because of CIHR funding.

Though he views these changes as positive, Moher says it’s too early to know if they will radically improve transparency in the research community. The degree of positive change will depend on the quality of the information submitted to registries. It is therefore important, he says, to monitor that data closely over the next few years. “This would be the worst time to let our attention down.”

Dr. An-Wen Chan, a medical policy analyst at the Mayo Clinic in Rochester, Minnesota, has also long advocated for greater transparency in clinical research. Chan, a Rhodes Scholar who studied medicine in Alberta, cites 3 reasons why registration is important: It would be unethical if the risks of human participation in trials were not balanced by contributions to public knowledge; doctors and policy makers can’t make informed health care decisions if negative trials are buried; and potential trial participants deserve all the information available about a given intervention before joining a study to test it.

Like Moher, Chan is encouraged by the recent surge in registrations but believes more work remains to be done. In an email to CMAJ, he writes: “It is clear that an effective compliance mechanism is necessary to ensure registration of all existing trials. Existing mechanisms have a limited scope of coverage. Global implementation of trial registration will also be important to address given the increasing number of trials conducted in low- to middle-income countries.”

Though aware of the challenges that remain, advocates of registration are pleased with a particular aspect of the recent US mandatory-registration law (referred to as FDAAA 801): clinical investigators must now submit basic results to a public database. “That’s a very big development, says Dr. Deborah Zarin, director of ClinicalTrials.gov. “There has never been a legal requirement to report results in a structured manner.”

Since the law came into effect, the number of weekly registrations in ClinicalTrials.gov has risen from about 250, Zarin estimates, to as many as 350. The database will prove particularly important in ameliorating the effects of publication bias because registration is independent of journal publication, she says.

But there is still uncertainty about the long-term effects. “How the public will use that information and how well it works remains to be seen.”

One significant shortcoming of the FDAAA 801 results database is that it won’t contain any information on trials completed before Sept. 27, 2007. That’s a big problem, because important data about thousands of drugs now on the market will remain private, says Dr. Erick Turner, an assistant professor of psychiatry at Oregon Health and Science University in Portland, Oregon. Turner, a former FDA clinical reviewer who has published articles on clinical trial registries in medical journals such as the New England Journal of Medicine and PLoS Medicine, claims a retrospective registry would be far superior. The FDA has a registry of all drug company trials for new drug applications but, unfortunately, there is limited public access to it. “It’s a bit spotty,” he says. “Sometimes you find the drug you’re looking for, and sometimes it’s not there.”

Even a more robust registry wouldn’t necessarily improve transparency, says Turner, if the information isn’t properly disseminated. Doctors are taught to form opinions of medical treatments based on peer-reviewed literature, he says, but the data about a particular intervention that makes it into the pages of a medical journal is rarely representative of all available data. Last year, in a widely publicized study, Turner and several colleagues found that 94% of the published studies for 12 antidepressants were positive, though only 51% of all the studies conducted were positive (N Engl J Med 2008;358[3]:252–60).

Though most of the fault for selective reporting lies with the clinical investigators, others share the blame. Peer-reviewers sometimes fail to check study protocols, says Turner, and therefore don’t know when researchers spin their analysis to match their results. Even if negative studies are reported, he adds, journal editors are reluctant to publish them. Instead, they often follow an informal policy that “this isn’t going to make the front page of The New York Times, so let’s publish a positive one instead.”

The suspicion that editors favour positive studies was confirmed in a Cochrane systemic review, which found that trials considered to have a “wow” factor were 1.78 time more likely to be published than those with negative or null results. The authors estimated that journal editors publish about 73% of positive studies, but only 41% of negative studies (Cochrane Database Syst Rev 2009 Jan 21;[1]:MR000006).

Another concern about registration is that only high-income countries are doing it reasonably well. Poorer nations, where more and more drug companies are migrating trials, tend to have inadequate oversight mechanisms.

But even in richer countries, registries are limited in scope. “Some important gaps remain,” says Dr. Davina Ghersi, coordinator of the WHO International Clinical Trials Registry, in an email to CMAJ. “For example, legislation usually covers drug trials but not others. How do we comprehensively capture non-drug trials on registries?”

Other changes Ghersi would like to see in the near future include health ministries becoming more engaged in trial registration, countries establishing stronger links between registered data and trial results, and clinical investigators being required to register trials to receive approval from ethics committees.

Ghersi also hopes that all the effort being put into registering clinical trials won’t be in vain. “Data should not be deposited and forgotten. They should be used to generate information that can help us make better health care decisions.”