Doctors searching for definitive answers to clinical problems in medical research are not only naive — they may be dangerous. After all, there are no absolutes in medicine.
Well, maybe one, according to Dr. Michael Allan, associate professor and director of evidence-based medicine at the University of Alberta in Edmonton.
“I thought that many of our outcomes in medicine were objective, but I’ve figured out there is really only one objective outcome, and that’s death. Everything else is open to interpretation,” Allan said Nov. 16 during his keynote address to the Family Medicine Forum in Toronto, Ontario.
Antiquated medical theories, funding and publication biases, arbitrary statistical cutoffs, a poor understanding of absolute risk, an excessive focus on surrogate markers — these are some of the reasons physicians should be wary of jumping to conclusions about a medical intervention after gleaning a few nuggets from the latest hot study.
“We look at something, we seem to have a little bit of understanding about what’s going on, and we make a bunch of leaps without confirming whether it really works or not,” said Allan, whose address was entitled “Baffled, Befuddled and Bemused: How not to get fooled again, and again, and,...”
The same holds true for meta-analyses, said Allan. An absolute conclusion can still be flawed, whether it is drawn from one study or 100. “It’s kind of like eating a stew and you’re not quite sure what the mystery meat was. Sometimes, it wasn’t chicken. We are really seduced by getting a single, true answer from a group of studies.”
One thing in particular that doctors should avoid is focusing too much on manipulating surrogate markers, such as cholesterol levels, rather than on clinical outcomes, such as heart attacks or death (the ultimate adverse effect). And yet this happens all too frequently, noted Allan. “How often do we have surrogate markers? And how often do we worry about them? All the time.”
But reducing the level of a surrogate marker is not always in the patient’s best interest, Allan said while telling a tale of “the surrogate heart.” When doctors noticed that the presence of abnormal heartbeats after a heart attack was associated with increased risk of death, they came up with a pharmaceutical solution that reduced the unwanted heartbeats by 80%.
“And all was good until, and this often happens, some troublemaker comes along and asks questions like this: ‘Are we really saving lives’?’” Allan told the forum, which is co-hosted by the College of Family Physicians of Canada (CFPC), the Ontario College of Family Physicians and the CFPC’s Sections of Teachers and Researchers.
“When someone bucks the status quo what do we do to them?” he added, pausing for a moment before displaying a slide of a noose, which drew laughter from the audience.
In fact, it turned out the treatment that so effectively reduced the surrogate marker was actually harmful to patients. “It was the standard of care, and we were killing one in every 21 people,” said Allan.
Another problem with research findings lies in their presentation to patients. There is big difference, for example, between how effective a medication is and how much risk is posed to a patient who doesn’t take it, said Allan. “You really don’t know where you are unless you understand where you started from. If I tell you something reduces your risk by 50%, that sounds really good, but what if you had almost no chance of getting it?”
Vaccines for shingles, Allan noted, can reduce the relative risk of getting the disease by a whopping 70%. But in some age ranges, the risk of getting shingles is only 2%, and the absolute benefit is only 1.5%. In other words, 60 to 70 patients would have to receive a vaccine to prevent one episode of the condition.
“It’s not that the vaccines don’t work. I’m not talking about that. I’m just saying the promotion of a number between 50 and 70 can be misleading for patients,” said Allan.
Another problem common in medical literature is that researchers tend to create arbitrary statistical cutoffs to determine if a treatment is effective, or if a risk factor is present. Again, this can be misleading. “We need to understand what is the real risk, not whether, in absolute terms, it’s yes or no,” said Allan. “We need scientists to start to back away from these absolute cutoffs they use so they can get their papers published, and that’s what a lot of this kind of thing is about.”
The many biases present in the scholarly process also skew the true value of research in any given field of medicine, Allan argued. Research findings can be affected by funders, while studies with negative results tend to get buried and medical journals typically leap at the chance to publish studies with positive results. “It’s not only that trials are being hidden. You are also taking good trials and publishing them again and again,” he said.
So what is a busy family doctor to do?
Remain skeptical. Seek out reliable sources of unbiased information. And above all, avoid regarding patients as biological warehouses of medical data, Allan said. “Treat patients first. Don’t worry about surrogate markers. Don’t worry about numbers and certainly don’t worry about theories of medicine.”