Simple presentation of test accuracy may lead to inflated disease probabilities

EDITOR—We found that conveying information on the accuracy of tests in non-technical language improved doctors' ability to estimate disease probabilities accurately.1 We investigated whether doctors might misuse such non-technical presentation when considering the probability of endometrial cancer in a patient with positive results on transvaginal ultrasonography.

We presented 263 general practitioners in Switzerland with a pre-test probability of 10%, information that the patient was aged 65, and a positive transvaginal ultrasound result. Ninety two general practitioners (group 1) received no information on the test's accuracy; 92 (group 2) were told that the sensitivity of the test was 80% and specificity 60%; and 79 (group 3) were told that a positive result is obtained twice as frequently in women with endometrial cancer as in those without the disease, reflecting a likelihood ratio of 2. The last two statements are numerically equivalent since the likelihood ratio equals sensitivity/(1-specificity).

The table shows that the degree of overestimation of diagnostic accuracy varied with the presentational format. As we found previously,1 almost half of the doctors did not change their probability estimates after they were provided with the patient's age.

We also found that the non-technical format resulted in 25 of the 79 general practitioners in group 3 (32% (95% confidence interval 22% to 43%)) multiplying their pre-test probability by exactly 2. This is theoretically incorrect since, for example, a likelihood ratio of 2 changes a pre-test probability of 40% to 57% only, not to 80%, which requires a likelihood ratio of 6. Unfortunately, in our study, this mistake helped those respondents who did not change their pre-test probability after being given the patient's age to get close to the correct value, changing 10% into 20%, corresponding to an attributed likelihood ratio of 2.25.

The table also shows the results after omission of these 25 doctors. The provision of some form of quantitative information still seems advantageous (contrast group 1 v groups 2 and 3; P=0.0216). However, all comparisons including group 3 are affected by this stricter analysis.

Framing the diagnostic information in the user friendly way that we used for the likelihood ratio may invite doctors to use simple arithmetic and might lead to grossly inflated inferences when pre-test probabilities are high or likelihood ratios are larger.