San Francisco Syncope Rule to predict short-term serious outcomes: a systematic review

Interpretation

We summarized the accuracy of the San Francisco Syncope Rule in predicting a serious outcome in patients presenting to the emergency department with syncope. In our analysis, the modelled probability of a serious outcome in patients with a negative score with this rule was 5% or lower (95% CI 1%–5%), based on prior probabilities of a serious outcome between 5% and 26% and a pooled sensitivity estimate of 87% and using the upper bound of the 95% CI for the estimate (i.e., a conservative interpretation of the data). Because of between-study heterogeneity, these summary results should be interpreted with caution.

Originally, the San Francisco Syncope Rule was derived in a prospective study involving all patients presenting to the emergency department with syncope, and it was thus conceptualized as a tool for initial risk stratification.3 Over the past few years, investigators have modified the original San Francisco Syncope Rule, as well as its role in evaluating syncope.4^,17 These modifications may have been an important cause of the heterogeneity of predictive accuracy in our systematic review. In their validation study, Quinn and colleagues4 focused on patients in whom a cause of syncope was not evident in the emergency department. They emphasized that the San Francisco Syncope Rule should be part of a thorough medical history and physical examination and should not prevent investigation for other associated symptoms.34^–36 We identified three studies27^,29^,31 that directly compared the “front-end” application of the rule for all patients with application of the rule only for patients without an evident cause of syncope. One of these studies31 reported better performance of the rule, whereas the other two studies27^,29 reported poorer performance of the rule in patients without an evident cause of syncope.

The management of syncope in the emergency department varies greatly among physicians and centres,37 although several evidence-based recommendations have been published in recent years.38^–40 Therefore, the population of patients without an evident cause of syncope may vary considerably according to the practices of local emergency departments, and judgment for generalization of the San Francisco Syncope Rule in this population remains uncertain. Nevertheless, restriction of application of the rule to patients without an evident cause of syncope after initial evaluation appears to reduce the prevalence of serious outcomes. According to Bayes’ rule, the post-test probability will be reduced accordingly, given that patient selection does not substantially affect performance of the San Francisco Syncope Rule. Our subgroup analyses confirmed a reduction in the modelled probability of a serious outcome, given a negative score, from 5% or lower to 2% or lower (using the upper bound of the 95% confidence interval), when the San Francisco Syncope Rule was restricted to patients without an evident cause of syncope.

We identified cardiac arrhythmia as an important issue and a potential source of heterogeneity. In their original publications, the authors of the San Francisco Syncope Rule specified that the result was positive if any nonsinus rhythm or any new changes were present on the electrocardiogram (ECG).3^,4 They recently clarified that the ECG criterion was not to be limited to a single evaluation but should include any available ECGs, as well as cardiac monitoring in the emergency department.17 It is likely that in studies performed after the original derivation study, cardiac arrhythmia registered during monitoring in the emergency department was inconsistently classified as a missed serious outcome.

Schladenhaufen and associates30 reported 17 cases of arrhythmia out of 23 serious outcomes missed in a retrospective cohort of patients aged 65 years or older. Birnbaum and coworkers29^,41 applied the San Francisco Syncope Rule to a minority population of African American and Hispanic individuals with a high admission rate (86%). They reported a sensitivity of only 74% (95% CI 61%–84%), missing a total of 16 serious outcomes, including 8 cases of arrhythmia. In contrast, Quinn and colleagues4 reported 23 cases of arrhythmia diagnosed after the initial visit to the emergency department, but they reported only one missed serious outcome. Finally, in the study by Thiruganasambandamoorthy and coauthors,31 four of the five cases with a missed serious outcome involved arrhythmia (for one of which the serious outcome was “return for admission to hospital”). With inclusion of abnormalities detected during cardiac monitoring in the emergency department, three of these arrhythmias would have been classified as positive with the San Francisco Syncope Rule (i.e., true positive), which would have improved the sensitivity from 0.90 to 0.96 (95% CI 0.87–0.99). However, which patients with syncope should undergo routine ECG monitoring in the emergency department and the duration of the monitoring period have been insufficiently specified for the San Francisco Syncope Rule, and further investigation is required.17

Validation studies have reported single cases of stroke, myocardial infarction, subarachnoid hemorrhage and other outcomes that were missed by the San Francisco Syncope Rule. Fischer and coworkers25 reported six strokes as missed serious outcomes. However, their study was published only as an abstract, and it is questionable whether the inclusion criteria of the San Francisco Syncope Rule were correctly applied and whether patients whose neurologic function did not return to baseline were excluded (see Box 1).

None of the validation studies applied the San Francisco Syncope Rule as originally outlined in the derivation study (Appendix 3, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.101326/-/DC1). Important factors other than arrhythmia that may explain the variable performance of the San Francisco Syncope Rule in these validation studies are differences in the definition of an abnormal ECG,42^,43 the professional role of the person in charge of interpreting the ECG, the definition of a serious outcome and the study design (prospective or retrospective). However, the small number of studies and the multitude of potentially confounding factors prevent a meaningful sensitivity analysis of the influence of these factors on accuracy of the rule.

Extension of the rule with predictors selected in recently derived scores may enhance its predictive accuracy. Levels of natriuretic peptide in the brain, oxygen saturation and chest pain are possible candidates that were relevant in the recently derived ROSE (Risk stratification Of Syncope in the Emergency department) rule.16 A list of predictors employed in other rules is presented in Appendix 5 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.101326/-/DC1).

In October 2010, Serrano and coworkers33 published a general systematic review and meta-analysis on clinical decision rules for syncope in the emergency department. They reported similar pooled accuracy estimates for the San Francisco Syncope Rule, with sensitivity 87% (95% CI 79%–92%) and specificity 48% (95% CI 38%–59%). These authors explored heterogeneity between studies through subgroup analysis related to study design, outcome period, definition of ECG, ECG determination (by caring physician v. others) and patient selection (stable v. unstable). They found no statistically significant explanation for the variability in performance of the San Francisco Syncope Rule.

We conducted this systematic review according to a prespecified protocol, with a strict focus on studies employing the San Francisco Syncope Rule. We identified relevant studies, reviews and correspondence using a comprehensive literature search in seven databases. In comparison to the the systematic review of Serrano and coworkers,33 we found three additional studies.1^,25^,26 However, the possibility of publication bias can never be fully excluded and is difficult to assess for our particular research question, because the use of funnel plots may be misleading in diagnostic or prognostic accuracy studies.44

We assessed possible biases with the Quality Assessment of Diagnostic Accuracy Studies instrument individually for each study and elaborated on inconsistent use of the San Francisco Syncope Rule in the process of evaluating syncope. Our critical review of individual study differences further elucidates between-study heterogeneity and provides important supplementary information that will allow clinicians to better understand how to optimally apply the San Francisco Syncope Rule. Furthermore, we employed modern statistical methods to provide easily interpretable negative and positive post-test probabilities.45^,46 In addition, we have provided a comprehensive overview of the components of serious outcomes that were missed by the San Francisco Syncope Rule.