Sources of bias for authors of clinical practice guidelines

Clinicians base their decisions on input from multiple sources: comfort levels with past decisions; advice from influential people such as teachers, residency supervisors and other respected clinicians; and published information such as primary research studies, editorials, commentaries, reviews and clinical practice guidelines.

Clinical practice guidelines are an important instrument for knowledge translation. They are frequently created by groups of individuals with a variety of methodologic and clinical backgrounds chosen to provide balance. They are published in high-profile locations and easily accessible in electronic format by clinicians and patients. Unlike commentaries, editorials and systematic reviews, practice guidelines must provide definitive recommendations for action. Finally, they frequently carry the “seal of approval” from a relevant clinical society. In all cases, opinions that convert data into recommendations require subjective judgments. Since these judgments are a human endeavour, they naturally leave room for error and bias.

The most widely discussed form of bias is financial. Choudhry and colleagues1 documented that authors of guidelines published in the 1990s had many financial relations with the manufacturers of pharmaceutical products that were being discussed in the guidelines. When surveyed, only 7% of those authors reported believing that those relations influenced their opinions — but 19% of them thought that their colleagues were so influenced. How important the nature and size of financial relations are in influencing opinions is still unclear.2 One interaction of particular interest to me is the use of medical advisory bodies by pharmaceutical companies to build relations with influential physicians. Although these bodies are formed to advise manufacturers, participation frequently seems to involve extravagant trips to exotic locations. While I am not aware of any empiric evidence to support my opinion, it seems clear to me that this type of relationship will later lead to a more favourable view of that manufacturer's products in the minds of advisory-body participants who also serve on guideline committees.

Other potential sources of bias receive comment much less frequently. Long service to government committees or private insurers imprints a different agenda on the minds of experts, for example — one that may include, among other perspectives, limiting costs. The group dynamic of such committees may lead to negative views about any new product intended to establish a revenue stream for a manufacturer.

A third source of potential bias includes the participants' previously established “stake” in an issue. Imagine, for example, that you are the principal author of a major randomized trial of a new clinical strategy that demonstrated a statistically significant benefit for patients. It is highly likely, if you sit on a guideline committee, that experience will dominate your views of the entire body of evidence.

A fourth form of bias comes from the way you make your living. A recent review article3 in the New England Journal of Medicine promoted positron-emission tomography (PET) scans for cancer assessment; of the 2 coauthors, 1 was a radiologist (who, I assume, reads PET scans). The article contained a statement that “No potential conflict of interest relevant to this article was reported.” Really?

Finally, there are the inherent influences that come from personal experiences. I recently asked a colleague which antibiotic he would prescribe for a common infection. He picked a relatively new and expensive agent. When I asked him why he would not choose an older, much less expensive drug, he said that it just “did not feel right” to pick that one when everyone else had abandoned it — though he knew that the evidence has not shown the new agent to be clinically better. He recognized his response to be somewhat irrational, but this form of bias is hard to fight.

In summary, the necessary human factor in making judgments based on multiple sources of data can result in biased recommendations. In the last few years, bodies that convene guideline processes have begun to recognize financial conflicts of interest. To improve the validity of their guidelines, they need to recognize all of the other sources of bias as well. In doing so, they will face the challenge of balancing the expertise of those with intimate knowledge, who are more likely to be subject to these forms of bias, with nonexperts who may have less knowledge but fewer of the influences that contribute to bias — no easy task. Although these other influences may be even more difficult to document and quantify than financial ties, they are no less important. Therefore, I recommend that guideline committees (and those who work on the methodologies they use) study the issue. Such action might further improve the rigour and transparency of the guideline process.