In the fall of 2008, retiree John Dennis sued Anapharm, one of Canada’s largest clinical research firms, alleging he’d been injured after participating in a clinical trial three years earlier. Dennis, of Drummondville, Quebec, ingested a trial drug as part of a study in the fall of 2005 and was rushed from the testing facility to the emergency room at McGill University Health Centre in Montréal, Quebec. He demanded $95 000. His case has been settled out of court.
Although Canada — and especially Quebec — is home to some of the busiest clinical research sites in North America, according to a study done by market research firm KPMG, no one knows for sure how often research volunteers suffer harm.
“It’s quite rare to see this kind of proceeding going to trial,” says lawyer Pascal Bouchard of the Montréal offices of the law firm Fasken Martineau. Canadian companies administering trials typically respond to legal claims with a monetary offer to avoid publicity and the settlements can involve nondisclosure agreements, making it very difficult to gauge the frequency of serious complaints, he says.
It’s equally difficult to obtain numbers from the public and private agencies that oversee human research subjects, such as research ethics boards (REBs) and Health Canada.
“This area is utterly opaque,” says Michael McDonald, the Maurice Young Chair of Applied Ethics at the University of British Columbia in Vancouver. In all research involving human subjects, even in studies that are working “reasonably well … you’d expect people are going to get hurt,” he adds. “It isn’t that it isn’t happening out there, it’s just that the information isn’t being collected.”
To McDonald, the contrast between the data available on this country’s actual guinea pigs and its human ones is striking. Institutions are obliged to report numbers of animals used to the Canadian Council on Animal Care but they aren’t obliged to report on humans, so no one can say how many people enter clinical research trials in Canada in any given year.
The lack of information alone makes it hard to say if the procedures designed to protect research subjects are effective.
Scientists in Canada and abroad have criticized the burgeoning national and international regulatory apparatus that surrounds clinical trials as stifling innovation and choking off trials without affording any benefit to research volunteers. The charges have spawned major initiatives in the United States and Europe aimed at systematically testing the regulations for oversight of trials.
For its part, Health Canada held stakeholder consultations on its clinical trials regulatory framework during 2006 and 2007 and, this year, has been sponsoring the development of national standards for research ethics boards via the Canadian General Standards Board.
But Canadian scientists who want to improve the trials bureaucracy meet a deaf ear, according to Dr. Salim Yusuf, chief scientific officer of Hamilton Health Sciences Centre at McMaster University in Hamilton, Ontario. There should be a “radical re-evaluation of existing trial guidelines,” wrote Yusuf, a leader of the international ‘sensible clinical trials’ group, in an editorial (Clin Trials 2008;5:40–8). The group’s efforts to change the rules face “a huge uphill struggle,” he added.
Also pushing for reform is the Clinical Trials Transformation Initiative (www.trialstransformation.org), at Duke University in Durham, North Carolina, a public–private effort jointly initiated with the US Food and Drug Administration in 2007. Judith Kramer, the initiative’s executive director, says the problem is that every time a clinical trial encounters trouble, bureaucracy grows. A clinical investigator “gets a warning letter [from a regulatory agency] and all of a sudden there’s five new standard operating procedures,” she explains.
Bouchard says that’s also true in Canada; if a problem arises, a research organization’s legal advisers are apt to call for better standard operating procedures (SOPs). So he tells trials sponsors to “set up a really good SOP in house” so that researchers know how to respond if a patient has an unexpected reaction.
Yet, as procedures aimed at protecting research participants have proliferated, they’ve made clinical trials cumbersome and costly. To spend less and save time, the pharmaceutical and contract research industries have taken an increasing proportion of trials offshore, and Canadian scientists say the same obstacles are preventing some investigator-driven projects from going forward.
It’s particularly difficult to gather data on the most severely ill patients, wrote Deborah Cook, another McMaster trials expert, while complaining that the rules and regulations blocking her research do “little to enhance the validity of the trials or the safety of participants” (Clin Trials 2008;5:61–9).
It’s “a lot of useless bureaucracy and … too many middlemen,” says Robert Califf, cochair of the Clinical Trials Transformation Initiative and Duke’s vice chancellor for clinical research. “Right now, most of the money is going to the middle people,” he says, referring to the monitors who are sent to trial sites to search out discrepancies in the data.
Kramer says procedures, such as trial monitoring, account for nearly a third of the financial resources of any clinical trial. A 2009 review of trials conducted by the US National Institute on Drug Abuse backs up that claim: monitors at the institute ran up a bill of about US$1500 for each site visit and conducted more than 100 site visits per trial (Clin Trials 2009;6:151–61).
Ultimately, the Clinical Trials Transformation Initiative hopes to measure the effectiveness of such monitoring via empirical investigations. “We need to do research on research, as opposed to people responding to an incident,” Kramer says. To that end, the group has launched several projects aimed at assessing the effectiveness of various forms of site monitoring. The first step, she says, has been “to just document the range of practices.”
Many larger research centres have monitors on site or nearby, ready to step in when needed. At the Hospital for Sick Children in Toronto, Ontario, for example, after researchers running a drug trial mistakenly enrolled ineligible children, on-site monitors spent nearly six months ensuring that none of the results from ineligible patients were included in the final analysis. No patients were harmed, says Richard Sugarman, chair of the hospital’s REB.
But at some major centres, administrators would scrap on-site monitoring programs in favour of “statistical process control” — a form of chart-driven auditing that originated with assembly lines. Also known as central statistical processing, the system analyzes site-by-site data to reveal sites with particular problems, then targets only the problem sites for visits.
Duke’s clinical trial teams began using statistical process control in 2001 and found that compared with traditional audits, they saved about 1000 hours per year and produced data with fewer errors (Clin Trials 2009;6:141–50).
Another idea for eliminating labour-intensive site monitoring is to ensure the overall quality of research at any given site by certifying the site as research-ready before a trial begins.
Yet even if empirical studies show that such alternative techniques are more effective than the standard model for monitoring, widespread change won’t come easily. Many in the clinical research field “are convinced that they can’t possibly assure [data] quality if they don’t look at every ‘i’ and be sure that it’s dotted,” says Kramer. Until recently, regulatory authorities have encouraged that sort of obsessive data-checking but that’s beginning to change. The best example of this change is in the reporting of harmful incidents.
In the acronym-rich trials business, a patient who is injured or dies as a result of a study drug has experienced an ADR (adverse drug reaction) or an SAE (serious adverse event) in the US or Canada, and a SUSAR (suspected unexpected serious adverse reaction) in Europe. Wherever it happens, investigators have to report it to multiple authorities, many of whom don’t know how to interpret the reports.
Cook dismissed many of the techniques for interpreting adverse events as “scientifically flawed and misleading.” Similar complaints were aired at a 2005 US public hearing when research ethics boards (known in the US as institutional review boards) described being overwhelmed by the volume of adverse event reports. Four years later, in January 2009, the FDA responded by issuing new guidelines for reporting adverse events. Most are not serious and need not be reported, the FDA wrote, since, as “isolated” events, their “implications for the study cannot be understood.” The guidance urges that events be reported in a grouped fashion and concludes that overreporting adverse events “rarely supports an [ethics board’s] efforts to ensure human subject protection.”
Health Canada has been assessing reporting of adverse events recently but hasn’t issued new guidelines. The department is continuing “to conduct policy and regulatory analysis on the issue … as well as engaging with our stakeholder groups,” says spokesman Gary Scott Holub.
Meanwhile, the Canadian Association of Research Ethics Boards issued its own guidance in August 2009, urging periodic summary reports. The group wrote that Canada’s current system for reporting adverse events in human subjects research “is not working, does not enhance participant protection, and in fact may be hindering the REB’s capacity to review and respond to safety issues in a timely fashion which ultimately may be harming research participants.”
Despite the litany of criticisms, clinical trialists readily accept one method for interpreting safety issues: data safety monitoring by a committee of experts.
Data safety and monitoring committees, also known as data and safety monitoring boards, have been called the “conscience” of a study: under blinded conditions, clinical investigators can’t know how patients are responding to a drug and rely on the committee to protect patients from a harmful product.
The committees have garnered a lofty reputation after stopping many trials midstream, including pivotal Canadian trials. Though stopping a trial can spark debate, if a data and safety committee advises it, medical researchers take action. “Theoretically, the investigators could refuse,” says Dale Williams, a University of Alabama statistician who has chaired and served on many such committees. “But then they’d have serious funding problems because the funding agency isn’t about to ignore the recommendations of the data safety monitoring board.”
Monitoring committees are labour intensive — members may have to review hundreds of pages of data for a data safety meeting and the meetings typically last several hours — so they aren’t used for smaller trials.
But other similar ideas are being tested. In its project on “improving the system of reporting and interpreting unexpected serious adverse events,” the Duke group plans to test a number of different models, including oversight by a data coordinating centre, or using central data analysis to permit real-time evaluation of adverse events.
European regulators also report difficulties in tracking adverse events. A European Commission survey assessing Europe’s Clinical Trials Directive concluded last spring that between 2000 and 2007, the mean number of adverse event reports received per year by “competent” European authorities rose from 297 to 5724 (www.efgcp.be/downloads/icrel_docs/Final_report_ICREL.pdf). Not surprisingly, both European ethics committees and sponsoring companies have complained that telling the committees about so many serious adverse reactions amounts to “useless notification,” that won’t help prevent harm to research participants.
The survey findings show that rules intended to harmonize research oversight throughout Europe and the United Kingdom substantially increased administrative burdens and costs, while causing major delays in getting trials off the ground. In response, researchers are recommending a risk-based regulatory approach so that academic institutions running mostly low-risk studies don’t face the same regulatory burden as companies testing novel, riskier drugs.
Meanwhile, in Canada, the draft second edition of the Tri Council Policy Statement: Ethical Conduct for Research Involving Humans contains similar recommendations for so-called “proportionate review.” But Canada lags behind Europe and the US in its efforts to document problems faced while keeping human volunteers safe.
The delay may be due to the paucity of information and the lack of transparency, says McDonald. “I think there’s a lot we don’t know here.”
Ironically, the bureaucracy constructed to keep enrolled volunteers safe is difficult to track and relatively unstudied — and it’s not clear if it works. Experts don’t expect much will change. “Something will have to happen for Ottawa to take this seriously,” said Richard Carpentier, executive director of the National Council on Ethics in Human Research, at a recent conference. He sounded worried by his own words.
Footnotes
-
Previously published at www.cmaj.ca