Bayesian assessment of adverse drug reactions

The results presented by Fran Paradiso-Hardy and colleagues1 are an excellent example of formal Bayesian causality assessment2 of a series of reported cases of suspected adverse drug reactions to ticlopidine. A sensible reader might ask a number of questions. For instance, wouldn't listing the reported reactions, which is the usual approach, be just as informative? If assessment of individual cases must be done, wouldn't expert opinion be just as good or better? And who needs these complicated mathematical calculations with odds ratios and probabilities when the basic data from the case reports are so scanty? Isn't it wrong to use numbers when we don't usually have data to back them up? And what do the final numbers mean? Does a 0.95 posterior probability mean that an adverse reaction to ticlopidine will occur in 95% of people? What does the difference between the prior and posterior probabilities mean? I will attempt to answer these questions.

Assessment of individual cases is helpful because it incorporates additional information about the pattern and context of the reaction to weigh the strength of the association in each case. In the ticlopidine example, it turns out that the evidence in most cases favours a reaction to ticlopidine. However, the evidence might have been against ticlopidine causation. A simple listing of reported reactions would not have shown the difference. Many of the “reactions” listed in books like the Compendium of Pharmaceuticals and Specialties3 may have nothing to do with the suspected drug because this causality assessment was never performed.

Yes, expert opinion can be very useful in assessing causality in suspected adverse reactions. However, unstructured expert opinion is not very reproducible and the component parts of the experts' opinions are unstated and unquantified.4 Other than by credentials and reputation, one has no basis for deciding whether one agrees with the opinion given. With a structured approach reproducibility improves.5 One also knows how the component parts were scored and could potentially disagree and revise the assessment accordingly.

It is a strange and counterintuitive practice to apply numbers to subjective judgements. The judgements in this case are measures of uncertainty. However, it turns out that there are sensible and nonsensible ways to combine judgements about uncertain phenomena, and probability theory provides both a language for expressing these judgements (a probability number from 0 to 1) and a way of combining them that avoids self-contradiction (the laws of probability). Most of us are familiar with the idea that if the chance of getting a tail on 1 toss of a coin is 0.5 then the chance of getting tails on 2 tosses is 0.25 (0.5 × 0.5). This is an application of the multiplicative law of probability. The Bayesian approach combines probabilities in exactly this way. Of course, with more probabilities being combined, it looks more complicated. But the greater complexity is a strong argument for an organized and coherent approach.

The best summary of the results in the paper by Paradiso-Hardy and colleagues1 is contained in Table 4. In the agranulocytosis column, the median posterior probability is 0.95. This is what might be called a retrodictive probability. The individual posterior probabilities answer the question “In this person taking ticlopidine in whom agranulocytosis developed, what is the probability that the agranulocytosis was caused by the ticlopidine?” The posterior probability gives some measure of the plausibility of ticlopidine causation in the individual case. It is not a predictive probability. It does not mean that agranulocytosis would be expected to develop in 95% of people taking ticlopidine. Notice also the difference between the prior (0.81) and the posterior (0.95) probabilities for ticlopidine and agranulocytosis. The prior probability comes from general epidemiologic information about agranulocytosis and ticlopidine that is separate from these cases. The posterior probability combines this background information (on the overall rate of agranulocytosis in people taking ticlopidine for this indication and in people like this not taking ticlopidine) with the evidence in the individual case to come up with an estimate of causation in each case. The difference between the prior and posterior probabilities is the contribution of the specific findings in each case. The background information clearly makes the biggest contribution to the final probability. With the background information alone, we are able to estimate that the median probability that ticlopidine caused agranulocytosis in the individual cases is 0.81 (prior probability). Additional specific case information increases this by a relatively small amount, to 0.95 (posterior probability). However, the case information helps to clinch the evidence. And the result could have gone the other way. If the specific case information had been against causation by ticlopidine, the posterior probabilities would have been lower than the prior probabilities, leading us to question the likelihood of causation in these cases.

My assessment is that the Bayesian approach is a real contribution to the evaluation of adverse drug reactions, and Paradiso-Hardy and colleagues should be congratulated for its use in their article.1 The reason that we do not use this method routinely in clinical practice is probably because it takes too much time and effort to be specific, clear and coherent. Reading a detailed analysis of a single case will make this point clear. For instance, in a case of acute renal failure possibly caused by gentamycin that I analyzed6 I had to think about the expected timing of acute renal failure if caused by gentamycin. I also had to express these expectations in quantified terms (1% on day 7, 3% on day 8, etc). But this was only one of many features I needed to identify and quantify: the other potential causes, their expected timing, the relevance of the urianalysis findings, the expected duration of the renal failure, the importance of pathologic data and so on. I needed to clarify and quantify all of these features before I could put the whole case together to arrive at a posterior probability for causation. This process took weeks rather the few minutes it might take an expert to arrive at a global assessment in this kind of case. The development of spreadsheet programs7^,8 and computerized expert systems9 will help, but we still have some way to go to make this approach generally applicable. But, in my opinion, the sooner the better!