Abstract
Background: The ABCD2 score (Age, Blood pressure, Clinical features, Duration of symptoms and Diabetes) is used to identify patients having a transient ischemic attack who are at high risk for imminent stroke. However, despite its widespread implementation, the ABCD2 score has not yet been prospectively validated. We assessed the accuracy of the ABCD2 score for predicting stroke at 7 (primary outcome) and 90 days.
Methods: This prospective cohort study enrolled adults from eight Canadian emergency departments who had received a diagnosis of transient ischemic attack. Physicians completed data forms with the ABCD2 score before disposition. The outcome criterion, stroke, was established by a treating neurologist or by an Adjudication Committee. We calculated the sensitivity and specificity for predicting stroke 7 and 90 days after visiting the emergency department using the original “high-risk” cutpoint of an ABCD2 score of more than 5, and the American Heart Association recommendation of a score of more than 2.
Results: We enrolled 2056 patients (mean age 68.0 yr, 1046 (50.9%) women) who had a rate of stroke of 1.8% at 7 days and 3.2% at 90 days. An ABCD2 score of more than 5 had a sensitivity of 31.6% (95% confidence interval [CI] 19.1–47.5) for stroke at 7 days and 29.2% (95% CI 19.6–41.2) for stroke at 90 days. An ABCD2 score of more than 2 resulted in sensitivity of 94.7% (95% CI 82.7–98.5) for stroke at 7 days with a specificity of 12.5% (95% CI 11.2–14.1). The accuracy of the ABCD2 score as calculated by either the enrolling physician (area under the curve 0.56; 95% CI 0.47–0.65) or the coordinating centre (area under the curve 0.65; 95% CI 0.57–0.73) was poor.
Interpretation: This multicentre prospective study involving patients in emergency departments with transient ischemic attack found the ABCD2 score to be inaccurate, at any cut-point, as a predictor of imminent stroke. Furthermore, the ABCD2 score of more than 2 that is recommended by the American Heart Association is nonspecific.
See related commentary by Hill and Coutts on page 1127 and at www.cmaj.ca/lookup/doi/10.1503/cmaj.110704.
There are approximately 100 visits to the emergency department per 100 000 population for transient ischemic attack each year.1 Although often considered benign, transient ischemic attack carries a risk of imminent stroke. Studies have shown that the risk of stroke is 0.2%–10% within 7 days of the first transient ischemic attack, and this risk increases to 1.2%–12% at 90 days.2–9 Stroke continues to be the leading cause of disability among adults and the third-leading cause of death in North America.10,11 Identifying people with transient ischemic attack who are at high risk of stroke is an opportunity to prevent stroke.3,4 However, urgent investigation of all transient ischemic attacks would require substantial resources. Three studies have attempted to develop clinical decision rules (i.e., scores) for assessing whether a patient with transient ischemic attack is at high risk of stroke.9,12,13 Combined, these studies led to the development of the ABCD2 (Age, Blood pressure, Clinical features, Duration of symptoms and Diabetes) score. However, despite its widespread implementation, the ABCD2 score has not yet been prospectively validated.12,14–18 This essential step in the development of rules for making clinical predictions has recently been requested.14,19–21
The objective of this study was to externally validate the ABCD2 score as a tool for identifying patients seen in the emergency department with transient ischemic attack who are at high risk of stroke within 7 (primary outcome) and 90 days (one of the secondary outcomes).
Methods
Study design and setting
Patients with transient ischemic attack who had visited one of eight Canadian emergency departments between 2007 and 2010 were eligible for enrolment in the study. The combined annual census at all participating sites was 394 000 visits.
Patient population
We enrolled patients, 18 years of age or older, who had received a final diagnosis of transient ischemic attack or minor stroke at the emergency department. Both patients with transient ischemic attack and those with minor stroke were enrolled because physicians could not observe patients for 24 hours to ensure the resolution of symptoms and thus distinguish between these two diagnoses.
Patients were excluded if stroke was confirmed at the time of assessment (i.e., neurologic deficit > 24 h), if they had a score of less than 15 on the Glasgow Coma Scale, if there was a documented alternative cause for their deficit (e.g., hypoglycemia, seizure, electrolyte imbalance or migraine) or if they were presenting to the emergency department more than seven days after their symptoms began.
The decisions to conduct investigations, refer to a specialist and start treatment were at the discretion of the treating physician. Physicians were explicitly advised not to alter the care they would usually provide because of their participation in this study. Patients were informed that they might be contacted by telephone, and verbal consent was obtained at the time of each contact. Each of the local research ethics boards approved the study.
Outcome measures
The primary outcome was stroke within seven days of the patient’s index visit to the emergency department. Secondary outcomes included stroke within 90 days of the index visit to the emergency department and recurrent transient ischemic attack within 7 or 90 days. These outcomes were assessed using hospital records from each site, including admission, clinic and autopsy reports. Patients assessed by a neurologist who received a final diagnosis of stroke or recurrent transient ischemic attack were deemed to have had that subsequent outcome.
Patients were contacted for follow-up by telephone at 7 and 90 days to verify the absence of outcomes using the Questionnaire for Verifying Stroke-Free Status.22,23 Patients were asked about subsequent visits to the hospital or to a physician for stroke or transient ischemic attack. An adjudication committee blinded to the documentation from the patient’s index visit reviewed all patients who answered “yes” to any question or for whom stroke or subsequent transient ischemic attack was diagnosed by a clinician who was not a neurologist. A positive outcome had to be confirmed independently by two of the three members of the committee.
Data collection
Data collection forms were completed by physicians during enrollment, before patients were discharged or referred to specialists. The data forms explicitly listed each component of the ABCD2 score (age ≥ 60 years [1point]; blood pressure ≥ 140/90 mm Hg [1 point]; unilateral weakness [2 points]; impaired speech without weakness [1 point]; duration of symptoms ≥ 60 min [2 points]; duration of symptoms 10–59 min [1 point]; diabetes [1point]) and indicated that a patient with a score of more than 5 was considered to be at high risk for subsequent stroke. Physicians recorded each of these five variables, calculated the total score and classified the patient as being at high or low risk. Physicians were also asked how comfortable they were with using the rule to indicate urgent (high risk) versus delayed care (low risk) for each patient using a five-point Likert scale. When feasible, a second independent physician, blinded to the first assessment, completed a separate data collection form. ABCD2 scores were also calculated by a trained research nurse at the coordination centre, blinded to outcomes, using the treating physician’s record of treatment and the study form. This score was termed the “coordinating centre score.”
Data analysis
The performance of the ABCD2 score for classifying risk was assessed with 95% confidence intervals (CIs) for sensitivity and specificity for subsequent stroke stratified by total score. The a priori high-risk cutpoint was defined as a score of more than 5 using the original work of Johnston and colleagues.12 We stratified the results to identify the optimal cutpoint and explicitly identified a score of more than 2 to be of interest. This decision was based on the guidelines of the American Heart Association for managing transient ischemic attack.24 The interobserver agreement between physicians was calculated using the intraclass correlation (ICC) coefficient. The score calculated by the enrolling physician was compared with the score calculated by the coordinating centre to estimate reliability. We generated receiver operating characteristic (ROC) curves for the scores given by both the coordinating centre and the treating physician. We calculated the areas under the curves, including the 95% CIs, for all outcomes.
Results
We enrolled 2056 patients, representing 72.9% of all eligible patients (Appendix 1, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.101668/-/DC1). The 766 missed eligible patients were similar to the patients in our cohort (mean age 68.8 [standard deviation (SD) 15.2 yr, range 19–98], 52.6% female). Telephone follow-up was successful for 1874 patients (91.1%) at or after seven days from the initial visit. Indirect assessments were available for 174 patients. Of these patients, 126 had a direct encounter with a neurologist or a subsequent admission to hospital, 3 patients died from stroke, 1 patient died from a nonischemic event more than 90 days after the initial visit to hospital, 1 patient died of unknown causes more than 7 days after the initial visit, 1 patient moved out of the country, 16 patients had subsequent visits to the hospital more than 90 days after the initial visit, 21 patients had subsequent visits to hospital more than 7 days (but less than 90 days) after the initial visit and 5 patients were contacted but declined to answer questions. Eight patients were lost to all follow-up at seven days. Because none of these patients returned to the regional neurologic centres, they were classified as not having had a subsequent stroke.
Table 1 summarizes the baseline characteristics of our cohort. The mean age of the patients was 68.0 years (SD 14.3) and half were women. Nearly half of the patients had motor weakness, about one-third reported language disturbance and most had symptoms that lasted for at least 10 minutes. This was the first transient ischemic attack for most of the patients. Almost all patients had a computed tomography scan of their head and an electrocardiogram as part of their work-up. Altogether, 38 (1.8%) patients had a stroke within 7 days of the index visit, and 65 (3.2%) patients had a stroke within 90 days. An additional 56 (2.7%) patients had a recurrent transient ischemic attack within 7 days; 145 (7.1%) patients had a transient ischemic attack within 90 days. Initial management of care was marginally more intensive for patients with “high-risk” ABCD2 scores (Table 2). Patients in the high-risk group were more likely to have computed tomography (p = 0.01), have a consultation with a neurologist in the emergency department (p = 0.001) or be admitted to hospital (p = 0.001). Only 75 (17.9%) patients with an ABCD2 score of more than 5 (i.e., patients at high risk of subsequent stroke) were admitted to hospital.
Physicians rated their level of comfort with using the ABCD2 with a score of greater than 5 indicating high risk as “very comfortable” for 15.5% of patients, “comfortable” for 46.4% of patients and “uncomfortable” or “very uncomfortable” for 13.3% of patients. Of the 23 patients who were given a score corresponding to a lower level of risk (i.e., ABCD2 ≤ 5) who subseqently had a stroke within seven days, physicians had indicated that they had neutral to very uncomfortable feelings with using the ABCD2 score to categorize risk for 10 of them.
The ABCD2 scores calculated by treating physicians at the bedside differed from the scores calculated at the coordinating centre, with physicians’ scores being lower for one-third of patients (Table 3; ICC 0.76, 95% CI 0.74–0.78). The most common error we found was that unilateral weakness was not scored if it was mentioned in the patient’s history but was no longer present on examination. At the cutpoint of more than 5, 154 (7.6%) patients were misclassified as being at low risk by treating physicians, and 37 (1.8%) patients were misclassified as being at high risk.
We included 39 patients for whom a second, independent physician assessed their complete ABCD2 scores. These scores had an ICC coefficient of 0.61 (95% CI 0.37–0.77). Overall, a “high-risk” ABCD2 score of more than 5 was insensitive for predicting stroke at seven days (sensitivity 31.6% (95% CI 19.1%–47.5%). The performance of the scores given by the coordinating centre at each cutpoint for stroke at 7 and 90 days is shown in Table 4. Using a cutpoint of more than 2 was highly sensitive but nonspecific (specificity 12.5% [95% CI 11.2%–14.1%]), with 87.6% of our patients classified as requiring urgent investigations or admission to hospital.
Table 5 shows the classification performance of the ABCD2 score as a predictor of recurrent transient ischemic attack. Again, scores with the highest sensitivity (e.g., ABCD2 scores of 3 or more) had low specificity. Table 6 summarizes the likelihood ratios for subsequent stroke or transient ischemic attack at 7 and 90 days for each given ABCD2 score. Although very low scores make a subsequent neurologic event unlikely, most of the likelihood ratios are close to 1.00, indicating little or no prognostic information.
Figures 1 and 2 show the ROC curves for the ABCD2 scores for the outcomes of stroke and recurrent transient ischemic attack at both 7 and 90 days. The score as calculated by the enrolling physician was poorly predictive for the primary outcome (area under the curve [AUC] 0.56 [95% CI 0.47–0.65]). The predictive value of the score as calculated by the coordinating centre was only slightly better (AUC 0.65 [95% CI 0.57–0.73]). We found similar results for the secondary outcomes.
Sensitivity analysis of the performance of the ABCD2 score as a predictor of stroke at 7 and 90 days was done for the subgroups of patients who had minor stroke and patients who had transient ischemic attack (i.e., the entire cohort less the patients with minor stroke). For the 228 patients with minor stroke, the high-risk classification of an ABCD2 score of more than 5 had a sensitivity of 50.0% (95% CI 18.8%–81.2%) for predicting subsequent stroke at 7 days and 53.3% (95% CI 30.1%–75.2%) for predicting subsequent stroke at 90 days. This is comparable with the sensitivity for predicting subsequent stroke among patients with “pure” transient ischemic attack (28.1% [95% CI 15.6–45.3] sensitivity at 7 days and 22.9% [95% CI 13.3–36.5] sensitivity at 90 days).
Interpretation
We found that an ABCD2 score of more than 5 had low sensitivity for predicting subsequent stroke at 7 or 90 days. We believe that the sensitivities we found are too low to be clinically acceptable.25,26 Although the score of more than 2 that is used to classify moderate-to-high risk (as suggested by the American Heart Association) did show sensitivity for predicting subsequent stroke, the specificity was poor, requiring almost all patients to have rapid investigations or be admitted to hospital.
We found that physicians frequently miscalculated the ABCD2 score and misclassified patients’ risks, despite calculating the score on a form that identified each component of the rule. The most common reason for underscoring was not allocating points for a history of unilateral weakness that had resolved at the time of the physical examination. This error might be mitigated by separating weakness into two separate variables, or by changing the name of the variable to something such as “unilateral weakness on history or examination.”
Comparison with other studies
The ABCD2 score was a combined effort by teams led by Johnston and Rothwell, who merged two separate datasets to derive high-risk clinical findings for subsequent stroke.9,12,13 Rothwell’s dataset was small, was derived from patients who had been referred by primary care physicians and used predictor variables scored by a neurologist one to three days later. Johnston’s dataset was derived from a retrospective study involving patients in California who had a transient ischemic attack.9
Subsequent studies evaluating the ABCD2 score have been either retrospective studies or studies using information from databases. Ong and colleagues found a sensitivity of 96.6% for stroke within seven days when using a score of more than two to determine high risk, yet 83.6% of patients were classified as high-risk.15 Fothergill and coworkers retrospectively analyzed a registry of 284 patients and found that a cutoff score of less than 4 missed 4 out of 36 strokes within 7 days.16 Asimos and colleagues retrospectively calculated the ABCD2 score from an existing database, but they were unable to calculate the score for 37% of patients, including 154 of the 373 patients who had subsequent strokes within 7 days.17 Sheehan and colleagues found that the ABCD2 score discriminated well between patients who had a transient ischemic attack or minor stroke versus patients with transient neurologic symptoms resulting from other conditions, but they did not assess the score’s predictive accuracy for subsequent stroke.18 Tsivgoulis and coworkers supported using an ABCD2 score of more than 2 as the cutoff for high risk based on the results of a small prospective study of patients who had a transient ischemic attack and were admitted to hospital.27 The systematic review by Giles and Rothwell found a pooled AUC of 0.72 (95% CI 0.63–0.82) for all studies meeting their search criteria, and an AUC of 0.69 (95% CI 0.64–0.74) after excluding the original derivation studies. The AUC in our study is at the low end of the confidence band of these results, approaching 0.5.28
Strengths
We conducted a large multicentre study prospectively validating the ABCD2 score using prospectively applied eligibility criteria and well-defined outcomes, which followed the methodological standards recommended for validation studies.19–21,29 The ABCD2 score was calculated by physicians in the manner in which the score is intended to be used (at the bedside, before subsequent events) and was centrally verified for accuracy.
Limitations
We used the traditional definition of transient ischemic attack, which is a clinical diagnosis based largely on history and without the benefit of magnetic resonance imaging to exclude small infarcts, consistent with current practice in most emergency departments.30 We expect that we enrolled some patients with neurologic symptoms with nonischemic causes (e.g., postural hypotension, migraine, psychogenic). However, our high proportion of patients receiving antiplatelet treatment, and the ABCD2 scores themselves, suggest that the participants resemble the population of interest.
The number of interobserver assessments was limited by the availability of a second eligible physician. We did not identify any systematic reason for a difference in those patients who received a second interobserver assessment; hence, we believe the data are accurate.
Despite our best efforts, not all eligible patients were enrolled in the study. We do not suspect any systematic reason for this other than the realities of doing research in busy emergency departments.
Physicians managed their patients’ care based on their usual practice and were certainly aware of their patients’ ABCD2 scores when making decisions. Increasing the awareness of transient ischemic attack as a neurologic emergency among both patients and physicians, ever-evolving approaches to same-day neuroimaging, prioritized carotid endarterectomy and intensified antiplatelet treatment among high-risk patients may all have contributed to our results. However, there was only moderate evidence of more intensive treatment among patients with higher ABCD2 scores. It is also unclear that the ABCD2 score per se is able to provide an accurate prognosis. Nevertheless, it is precisely because practice evolves that rules for making decisions must be validated in a contemporaneous setting.
Clinical implications
The ABCD2 score can be credited for increasing awareness of transient ischemic attack as a medical emergency that carries with it a substantial and modifiable risk for subsequent stroke. This study determined that the criteria used to calculate the score are not sensitive enough to classify patients as being at low risk. We also determined that the 2009 recommendation by the American Heart Association of using an ABCD2 score of more than 2 to determine high risk has a very low specificity, classifying almost all patients as requiring immediate imaging and perhaps admission to hospital.
Implications for research
Transient ischemic attacks are common, and strokes are best prevented, but rapid investigations require many resources. Prioritization and risk stratification of patients is therefore essential. A prospective multicentre study following the established guidelines to develop a clinical decision-making rule with greater sensitivity, while maintaining the highest specificity possible, is required. Additional research is needed to determine the optimal interventions for high-risk patients to avoid stroke that have the least impact on available resources. This is especially true in countries with low or medium incomes, where the incidence of stroke has increased by 100% over four decades, as they have limited access to advanced imaging and likely cannot afford to admit most patients who have transient ischemic attacks to hospital.31
Conclusion
This prospective study found that the high-risk ABCD2 score is not sensitive enough to be the sole guide for assessing risk for patients in emergency departments with transient ischemic attack. Using the current recommendation of the American Heart Association (i.e., an ABCD2 score of more than 2) does not discriminate well between patients at high and low risk, thus requiring urgent care for nearly all patients and thereby limiting its clinical utility.
Acknowledgements
Dr. Perry is supported by a Canadian Institutes for Health Research New Investigator Award and was previously supported as a Career Scientist by the Ontario Ministry of Health. Dr. Stiell is a Distinguished Professor and University Health Research Chair, University of Ottawa. The authors thank the hundreds of physicians who completed data collection forms and all of the emergency department nurses and clerks at the eight study sites for their cooperation with the study. The authors also thank the following research personnel at the study hospitals: Ottawa Hospital — Civic Campus and General Campus, Ottawa, Ont. (Rebecca Briscoe, Juanita Wilzer); Kingston General Hospital and Hotel Dieu Hospital, Kingston, Ont. (Kathy Bowes, Jane Reid, Nicholas Martin); Hôpital de L’Enfant-Jésus, Québec City, Que. (Marilyne Dufresne); Hamilton Health Sciences Centre — General, McMaster and Henderson Sites, Hamilton, Ont. (Christina Brean). The authors thank their colleagues at the Ottawa Hospital Research Institute (Sarai Cohn-Kalter, Malaika Mvungi, Sheryl Domingo, My-Linh Tran, Irene Harris, and Angela Marcantonio) for their assistance with this project.
Footnotes
-
This article has been peer reviewed.
-
Competing interests: Jeffrey Perry, Mukul Sharma, Marco Silvotti and Andrew Worster have received grants from the Canadian Institutes of Health Research. Mukul Sharma has received a grant from the Heart and Stroke Foundation. He is a consultant for the Canadian Agency for Drugs and Technologies in Health and Talecris, is employed by the Canadian Stroke Network, has received payment for lectures from Sanofi and Bristol-Myers Squibb, Boehringer Ingelheim, Pfizer and Merck and has received payment for the development of educational presentations from Boehringer Ingelheim. Ariane Mackey has received payment for lectures from Boehringer Ingelheim and been reimbursed for meeting expenses by Sanofi–Aventis and Bristol-Myers Squibb. Wieslaw Oczkowski has received payment for lectures from Boehringer Ingelheim. Steve Verreault has received payment for lectures from Sanofi and has been reimbursed for travel expenses by Boehringer Ingelheim and D-Pharm. George Wells is a biostatistical consultant for the CMAJ and was not involved in the editorial decision-making process for this article. No other competing interests were declared.
-
Contributors: Jeffrey Perry conceived the idea and prepared the manuscript. Cheryl Symington and Jane Sutherland coordinated the study, collected data and contributed to the writing of the manuscript. George Wells provided considerable statistical assistance and revised the manuscript. Mukul Sharma and Ian Stiell provided much input into the study design and the revision of the manuscript. Marco Sivilotti, Marcel Émond, Andrew Worster, Grant Stotts, Albert Jin, Weislaw Oczkowski, Demetrios Sahlas, Heather Murray, Ariane MacKey and Steve Verreault assisted with the study design and revised the manuscript.
-
Funding: The authors would like to acknowledge peer-reviewed funding from the Canadian Institutes for Health Research (grant MOP 84372).