During a 2016 department research retreat in Ontario, a medical school professor described cases of research fraud that had received international attention. Several students came up afterward to say they connected personally to his topic. During discussions with their research supervisors, they “felt an unspoken expectation to ensure their data fit with the hypothesis,” said the professor. “Like, if there are any outliers, get rid of it, that kind of thing.”
The professor, who declined to be named to protect the students’ identities, was surprised that he’d struck a chord. But surveys over the past several years show it’s not rare for scientists to cut corners in their work. In a 2009 meta-analysis of 18 large surveys, Daniele Fanelli of Stanford University found that up to 34% of scientists — including medical researchers — admitted “dropping data points based on a gut feeling” or other questionable research practices, and as many as 72% had seen questionable behaviour by a colleague.
A survey of 2155 research psychologists published in 2012 concluded that the majority were involved in practices such as “deciding whether to exclude data after looking at the impact of doing so on the results.”
The problem of scientists showing only the data that support their case “is absolutely, extremely prevalent,” said Dr. Gary Lewis, an endocrinologist and director of the Division of Endocrinology and Metabolism at the University of Toronto. Lewis started thinking about the idea that scientists can go down a slippery slope towards misconduct when a member of his division, Dr. Sophie Jamal, was found guilty of manipulating data. “I think it starts small, with not being true to the data, and I think you can take one step and then another step and another step, and before you know it, you’re in over your head,” he said.
Researchers involved in questionable practices “probably are quite fearful and have an understanding there’s a better way to do it,” said Nancy Walton, chair of the Research Ethics Board (REB) at Women’s College Hospital in Toronto. She thinks scientists might take a different path if offered support at the right time, but there isn’t a system to provide that sort of guidance, and research authorities are seen as punitive. “No one goes to an REB [for research advice],” she said. “They’re terrified of REBs, and they’re not going to go to their VP of research.”
Education is typically seen as the answer. In response to a series of misconduct cases, the University of Toronto’s medical school established a research integrity task force. Its “first goal is to really educate our faculty” about ethics guidelines and expectations, said task force chair Dr. Allan Kaplan, a vice-dean at the school.
Training in research integrity builds participants’ knowledge, but the effects don’t last long, according to a 2016 Cochrane Collaboration review. The review also concluded that effects of training on reducing misconduct “are uncertain” because of “very low quality” evidence.
CMAJ has learned that Jamal had completed all the mandatory training at her hospital in good clinical practice, research ethics, and responsible conduct of research during the period when she was found to have made changes to her data.
A surer bet for discouraging bad research behaviour may be greater institutional support, including information technology support for electronic data capture platforms and plagiarism detection software. By increasing the chance that misbehaving scientists will be found out early on, such systems could possibly deter misconduct.
Dr. Richard Eastell, an osteoporosis expert in England who collaborated with Jamal on a paper that he and other authors retracted after her hospital’s investigation found the data were unreliable, said preventing similar situations in the future requires institutional processes to monitor data throughout the course of a study.
Universities and research centres across Canada offer systems, such as REDCap, that track who enters original data points or alters them over the life of a study, but researchers require the infrastructure to use such systems, including access to experienced programmers. Monitoring visits and random research audits may also catch problems.
Health Canada requires onsite monitoring of the studies it approves. Drug companies also monitor studies they sponsor and a few of the largest institutions now monitor studies by their scientists. But many academic researchers running so-called investigator-initiated studies “don’t have the funds to do that kind of monitoring so they may not do it at all, or they may not do it as frequently as they should,” said Karen Arts, executive director for the Canadian Cancer Clinical Trials Network.
To catch plagiarism, many journals use CrossCheck from iThenticate, and some institutions, including the University of Waterloo and Toronto’s St. Michael’s Hospital, now provide free access to iThenticate so scientists can check articles before submitting them.
As for the students who spoke up at the annual retreat, the professor told them how to report their supervisors, but he understood their reluctance. Students’ careers depend on letters of reference and networking which could be “gone in a flash” if they lose the backing of a mentor, he said.
Walton said research assistants sometimes bring their concerns or questions about studies to the REB office because they “don’t have someone to turn to.” In her view, those situations speak to a “huge gap in the system”: the lack of structures to support researchers and their staff.