See related research article by Bösner and colleagues, page 1295
In this issue, Bösner and colleagues propose a rule to assist primary care practitioners in deciding whether a patient’s chest pain is due to coronary artery disease or not. 1 Evaluating these patients in primary care is not straightforward. The prevalence of serious disease is low overall, yet there are potentially life-threatening causes of chest pain that should not be missed. Over a quarter of patients who are admitted to hospital with acute myocardial infarction have had a related appointment with a primary care physician in the previous month; half of these patients may not be appropriately managed, in part because of the difficulty in diagnosing the cause of the chest pain. 2 The challenge is correctly identifying and referring the patient with coronary artery disease while minimizing the number of referrals of patients who do not have the disease, to reduce the harm of unnecessary investigation and the burden on health care resources.
Several existing decision rules for diagnosing coronary artery disease have been developed in secondary care. 3 In general, diagnostic rules for use in high-prevalence settings have lower predictive value in primary care. 4 For coronary artery disease, this would lead to more referrals of patients without the disease. As all health care systems seek to improve quality while containing costs, the search for appropriate prediction rules to support decision-making in primary care is timely.
The decision rule proposed by Bösner and colleagues is based on five simple factors ascertained during the consultation. The rule was derived and tested using data for 1199 patients who presented with chest pain in 74 primary care practices in Germany. It was further tested and validated using data for 672 patients who presented with chest pain in primary care practices in Switzerland. In both cohorts, the number of factors present correlated well with the odds of coronary artery disease being diagnosed.
To be ready for use in primary care, the decision rule needs a threshold for referral or ruling out of disease. A high threshold would give a high positive predictive value (most patients with suspected coronary artery disease would be found to have the disease) but a lower sensitivity (many patients would be erroneously ruled out). Lower thresholds would have improved sensitivity but a reduced positive predictive value. Bösner and colleagues suggest ruling out coronary artery disease when fewer than three of the five factors are present.
All of the considerations of diagnostic research apply to diagnostic decision rules, including appropriate sampling, good ascertainment of outcomes and careful selection of thresholds. In addition, many examples exist of decision rules that perform less well when tested with new data than when tested with the data from which they were derived. A credible decision rule should therefore be tested both in the sample in which it was derived (the “derivation cohort”) and in a further sample representative of the setting and population in which the rule is to be applied (the “validation cohort”). 5 Another challenge is to maintain usability. Some decision rules that have claimed high accuracy require dozens of input variables. This feature potentially limits feasibility in a short clinical consultation and the number of patients in whom all variables are available. 6,7
The study by Bösner and colleagues is in many ways a model of prediction-rule development. The collaboration with the Swiss research group has provided a solution to the difficulty that many experience in finding an appropriate cohort for validation. Both the derivation and the validation cohorts were large, relevant to the population in which the prediction rule would be applied, and designed to ensure comprehensive detection of coronary artery disease. Also, the rule achieved usability without compromising on predictive power. Having begun with a 14-variable regression model, the authors were able to reduce it to five key variables with minimal impact on accuracy. The resulting “any three of five” rule could hardly be easier to use, and — almost unique in prediction-rule research — there was no loss of predictive power in the validation cohort. It is instructive to compare this with the companion paper, 8 in which a comparatively complex risk score derived from the Swiss data was less successful in external validation. This may illustrate Dawes’ heuristic argument that the simplest decision rules are also the most robust. 9
An alternative explanation for the rule’s success in both the validation and derivation cohorts reported by Bösner and colleagues may have been the authors’ inclusion of patient perception as a variable. Different measures of patient perception were used in the two cohorts: “Patient assumes pain is of cardiac origin” in the derivation cohort and “Are you feeling very worried about your chest pain?” in the validation cohort. Although the use of different measures was not ideal, both questions probably addressed something similar: that the patient recognized his or her chest pain to be different from noncardiac pains experienced previously. Patient-reported measures are traditionally treated with suspicion in epidemiology; however, subjective measurements have shown surprising diagnostic power in other clinical areas in recent studies. 10
Is the decision rule developed by Bösner and colleagues ready for use? The positive predictive value of 40% at the proposed threshold is probably an improvement on current practice in many settings. Whether the sensitivity of 87% is acceptable depends on the nature of the 13% (95% confidence interval 6%–20%) of patients with coronary artery disease who are missed. If these patients have mild disease, delayed referral would have no prognostic impact; however, if they have more severe disease, delayed referral can lead to poorer cardiac outcomes or death. It is plausible that patients with fewer than three of the five factors have less severe coronary artery disease than average, and that the decision rule can be safely applied with appropriate diagnostic safety-netting. 11 Further research is needed to confirm this supposition and to examine the transferability of the results to other countries, health care systems and ethnic groups.
-
Primary care practitioners need to be able to correctly identify and refer patients whose chest pain is due to coronary artery disease while reassuring those who are very unlikely to have a serious underlying condition.
-
A decision rule developed and validated in two large cohorts shows remarkable promise in improving the diagnosis of coronary artery disease in primary care.
-
Further research is needed to confirm that the rule can be applied safely in primary care settings.
Key points
Footnotes
-
Previously published at www.cmaj.ca
This commentary was solicited and has not been peer reviewed.
Competing interests: Richard Stevens received speaker fees and travel assistance from the University of Marburg for a seminar presentation on methods for meta-analysis of individual patient data. No competing interests declared by Daniel Lasserson.
REFERENCES
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
- 9.
- 10.
- 11.