How can the scientific community ensure the integrity of COVID-19 research?

Recent clinical trials have shed new light on treatments for coronavirus disease 2019 (COVID-19), but retractions of COVID-19 papers have also sparked concerns about data quality and “science by press release.” To weigh in on these developments, CMAJ reached out to infectious disease specialists Dr. Srinivas Murthy of BC Children’s Hospital and the University of British Columbia in Vancouver, Dr. Matthew Oughton of Jewish General Hospital and McGill University in Montréal, and Dr. Alon Vaisman of the University Health Network in Toronto.

CMAJ: What has been the most interesting development in Canada’s pandemic response in recent weeks?

Oughton: The parallel stories of Ontario and Quebec. Both provinces topped the list of new COVID-19 cases for several weeks. Both instituted initial reopening measures almost simultaneously, and both have seen dramatic reductions in their numbers of new cases since reopening began. However, the differences between their monitoring strategies during reopening are striking; Ontario is reaching record numbers of tests per day (above 20 000), while Quebec is substantially below their target of 14 000. The next few weeks should indicate whether one province has chosen a better strategy.

Vaisman: It looks like Ontario and Quebec are on the same track as other provinces; it just took them longer to get there. There have been improvements in contact tracing in Ontario, and testing criteria expanded in late May, which may result in an initial bump in [confirmed] cases. Still, the ultimate effect will usually be a reduction [in new infections].

The COVID-19 pandemic has exposed gaps in research infrastructure and publication models.

Image courtesy of iStock.com/tuachanwatthana

Murthy: At the same time, people seem to be more and more complacent. The collective response has been, “We need to reopen as fast as possible,” and I think that’s problematic. Even in British Columbia, where there is relatively low community spread, we’re still having small-scale institutional spread.

CMAJ: How should practice change considering the recent results of clinical trials on treatments for COVID-19?

Oughton: Hydroxychloroquine looks to have little promise for postexposure prophylaxis, and its use in treatment seems increasingly in doubt. The study from Grein et al. in NEJM on remdesivir has raised some hope for its use in severe COVID-19 to reduce the duration of hospitalization, although its small size and methodologic limitations indicate that larger prospective, double-blinded, randomized control trials are needed to characterize its role in treatment better. The recent press release [followed by publication on a preprint server] on dexamethasone offers some exciting possibilities for an inexpensive and widely available medication.

Murthy: The dexamethasone results are quite exciting. If we’re able to reduce mortality by 30% with something cheap that we all have on our pharmacy shelves, that will have an incredible impact on the pandemic, more so than any of the fancy stuff that drug companies are investigating.

[Meanwhile,] the signal seems to be that there is no benefit from hydroxychloroquine, which is unfortunate. We have wasted three months, spent millions of dollars doing randomized trials and exposed thousands of patients to a drug that no one outside of a small number of people was convinced would be effective. But we had to prove it because of the rumours and conjecture.

In terms of remdesivir, the jury is still out. One study showed time to recovery benefit. But we’re not locking down our economies to [spend] two days less in the hospital. We’re locking down to prevent deaths, and we need drugs that can do that.

CMAJ: How can the scientific community ensure the integrity of research isn’t compromised in the rush to publish?

Oughton: Authors should avoid predatory journals in favour of journals with established reputations that follow standard peer-review processes. Journal editors and boards need to vet their reviewers and allow reviewers sufficient time to thoroughly evaluate manuscripts instead of setting all-too-brief deadlines for review of submissions. Finally, it is up to us, as the consumers of medical literature, to make it clear that our needs are not well served by reading pre-prints and manuscripts that are rushed through review processes.

Vaisman: There are complaints on both sides without any solutions. It’s unethical to hold onto results while people die, and [some might argue] it’s unethical to release results that are not peer-reviewed, or maybe people overestimate the importance of peer review as if it’s this magical antidote against dishonesty. If there was some level of verification that didn’t require the time-consuming process that goes into peer review, maybe that’s one way to mitigate the problem. [That might look like] a third party that verifies data very quickly or goes through a checklist of things that need to be met before research can go to preprint. Simply saying preprints are bad is not a solution, and neither is just throwing everything out there in public.

Murthy: I think we can rely on transparency [to ensure the integrity of research]. If someone wants to be fraudulent, they can get through [peer review] without much difficulty. There’s a certain amount of trust that we must put in somebody along the way, whether that trust is in the researcher, a university, a journal, or the media. [In the case of the recent dexamethasone press release], I know the researchers and collaborate with them frequently, so I can trust that they will report good data. And I think that concept where we build trust by experience and eminence can’t be ignored.

CMAJ: What has been the experience at your institution with clinical trials related to COVID-19? What challenges do you foresee Canadian hospitals encountering?

Vaisman: At the beginning of the pandemic, so many different researchers wanted to get trials done. My institution specifically set up a board to ensure that the highest quality studies were approved because very quickly when a patient came in with COVID-19, you had study coordinators lining up outside the patient’s room.

Oughton: The smaller population bases of many Canadian centres can make enrolment challenging, particularly when numbers of new cases are currently decreasing. The other major challenge is that access to medications from pharmaceutical companies on compassionate release grounds can be difficult if [the companies] do not have established facilities in Canada. For example, the approval process for medications used in severe illness may take several days, by which time the patient’s status may have changed.

Murthy: Canada has problems with clinical trial infrastructure. If you compare us to the United Kingdom, they have a central funder that supports hospitals in doing research and making that part of care. Ethics approval is all centralized. And there’s a priority setting process in other countries that makes things as coordinated as possible. We don’t have that in Canada. [We] fund individual projects, research coordinators and researchers, separated as much as possible from the care that’s provided. I think that model is a failing one. If you look at the proportion of patients hospitalized with COVID-19 in Canada who were in a clinical trial, it’s much less than in other countries.

CMAJ: Doctors at community hospitals have pointed out that most people with COVID-19 are managed in non-academic hospitals — how should they be supported to enroll patients into trials?

Murthy: In one of my trials, I’m specifically targeting community sites because I know that’s where the patients are, and that’s where the capacity needs to be built. We need the government to see this is important. We need the health system to see that research is part of care and how you learn to improve care. That involves funding research at the community level.

Oughton: One of the recent highlights in enrolling patients during COVID-19 restrictions came from Boulware’s paper in NEJM on hydroxychloroquine for postexposure prophylaxis. Centres from three Canadian provinces were involved. Enrolment, informed consent, and data collection were performed using remote methods, including Internet, telephone, and text message during times when routine methods such as in-person visits to hospitals were necessarily restricted. These kinds of innovative methods are worth considering for non-academic centres that may cover geographically large areas with low population densities, both during the current pandemic and beyond.

CMAJ: What has COVID-19 taught us about infection control beyond the lessons learned from SARS and other outbreaks?

Vaisman: One of the important lessons for infection control is that we need to have personal protective equipment (PPE) made in Canada, available to everyone, that can be ramped up at a moment’s notice. We just didn’t have PPE available, not to the point where we had confidence in the supply. Just like infection control, it falls by the wayside except when there’s a crisis.

Murthy: We’ve learned a lot about how much of infection control is behavioural. You really need to focus on knowledge sharing. Where messaging has been chaotic, infection control policies have been chaotic accordingly.

Oughton: I believe we will see gradual but increasing societal expectations for people to engage in measures that reduce risk to others, including frequent hand hygiene, a continuation of physical distancing in many public settings, and the wearing of non-medical masks. Our COVID-19 experience also taught us about the limitations of relying on prior experience with other pathogens when it came to issues such as asymptomatic transmission. Going forward, I hope we will approach new pathogens with the recognition that we need accurate and rapid research to provide a solid foundation for public health and infection control interventions.