Diagnostic accuracy and reproducibility in the interpretation of Ottawa ankle and foot rules by specialized emergency nurses

doi:10.1016/j.ajem.2005.02.054

The American Journal of Emergency Medicine

Volume 23, Issue 6, October 2005, Pages 725-729

https://doi.org/10.1016/j.ajem.2005.02.054 Get rights and content

Abstract

Objectives

The ED is often confronted with long waiting periods. Because of the progressive shortage in general practitioners, further growth is expected in the number of patients visiting the ED without consulting a general practitioner first. These patients mainly present with minor injuries suitable for a standardized diagnostic protocol. The question was raised whether these injuries can be treated by trained ED nurses (specialized emergency nurses [SENs]). The aim of this study was to evaluate the diagnostic accuracy and reproducibility of SENs in assessing ankle sprains by applying the Ottawa Ankle Rules (OAR) and Ottawa Foot Rules (OFR).

Methods

In a prospective study, all ankle sprains presented in the ED from April to July 2004 were assessed by both a SEN and a junior doctor (house officer [HO]) randomized for first observer. Before the study, SENs were trained in applying OAR and OFR. In all patients, radiography was performed (gold standard). The diagnostic accuracy for the application of OAR and OFR was calculated for both groups and was compared using z statistics. Furthermore, from the paired results, reproducibility was calculated using κ statistics.

Results

In total, 106 injuries were assessed in pairs, of which 14 were ultimately found to concern acute fractures (prevalence, 13%). The sensitivity for the SEN group was 0.93 (95% confidence interval [CI], 0.64-1.00) compared with 0.93 (95% CI, 0.64-1.00) for the HO group (no significance [ns]). The specificity of the nurses was 0.49 (95% CI, 0.38-0.60) compared with 0.39 (95% CI, 0.29-0.50) for the doctors (ns). The positive predictive value for the SEN group was 0.22 (95% CI, 0.13-0.35) compared with 0.19 (95% CI, 0.11-0.31) for the HO group (ns). The negative predictive value for the nurses was 0.98 (95% CI, 0.87-1.00) compared with 0.97 (95% CI, 0.84-1.00) for the doctors (ns). The interobserver agreement for the OAR and OFR subsets was κ = 0.38 for the lateral malleolus; κ = 0.30, medial malleolus; κ = 0.50, navicular; κ = 0.45, metatarsal V base; and κ = 0.43, weight-bearing. The overall interobserver agreement for the OAR was κ = 0.41 and κ = 0.77 for the OFR.

Conclusion

Specialized emergency nurses are able to assess ankle and foot injuries in an accurate manner with regard to the detection of acute fractures after a short, inexpensive course.

Introduction

Emergency departments are confronted with progressive crowding during rush hours. Because of the growing shortage of general practitioners, many patients tend to bypass the general practitioners office and come to the ED without referral [1], [2]. Moreover, increased patient demands on (emergency) health care provision also add to the ED workload. Most of the presentations concern minor injuries, such as ankle sprains, which when crowded, have to wait for a long time to be assessed and treated by the attending doctor. The waiting period is often prolonged because of the triage system, which puts more severe injuries first.

Several solutions have been proposed and introduced, of which the physician assistants (PAs) and emergency nurse practitioners (ENPs) are the best-known examples [3], [4], [5], [6], [7]. These methods have been proven successful in several investigative settings [8], [9], [10], [11]. However, there are disadvantages to the deployment of these midlevel practitioners. Firstly, the educational period is approximately 2 years, in which the nurses are less available for clinical work. After graduation, the midlevel practitioners are only deployed to perform diagnostic and logistic tasks, necessitating the need for an extra regular emergency nurse to perform the nursing tasks. Furthermore, the costs for education and salaries are high in comparison with regular emergency nurses [12].

In this context, the need to investigate the possibility of deploying regular emergency nurses to assess and treat certain injuries after specific training and according to a standardized protocol was established. The injury chosen to test this concept on is the ankle sprain because it is a common, well-defined injury for which the Ottawa Ankle Rules (OAR) and Ottawa Foot Rules (OFR) were developed to indicate whether a foot and/or ankle x-ray is needed [13], [14]. Since their introduction in 1992, these rules have been studied and validated extensively [15], [16], [17], [18]. However, little is known about the diagnostic accuracy and reproducibility of emergency nurses in interpreting the OAR and OFR [19], [20], [21]. Only a few studies have been conducted in which the accuracy and interobserver agreement of these interpretations were subject of research, mostly dealing with midlevel practitioners [22], [23], [24].

Therefore, the aim of this study is to assess the diagnostic accuracy and reproducibility of emergency nurses compared with junior doctors (house officers [HOs]) working in the ED in interpreting the OAR and OFR.

Section snippets

Study design

A prospective study was performed from April to June 2004, in which all consecutive ankle sprains of patients aged 18 to 65 years were included. The study was conducted in an urban university teaching ED with an annual patient census of 35 000. Each injury was assessed by both a trained emergency nurse (specialized emergency nurse [SEN]) and an HO by means of the OAR and OFR (Fig. 1). Randomization for first observer took place to prevent a hypothetical influence of the first assessment on the

Results

In total, 108 patients were assessed by 2 observers as described before. Of these 108, 2 were excluded afterwards because they were found to exceed the upper age limit. In both excluded patients, clinical investigation results of both observers were in accordance with each other and the gold standard. In total, 106 injuries were included in the study, of which 14 were ultimately judged by the radiologist to concern acute fractures (prevalence, 13%). Of these fractures, 5 were located in the

Discussion

The clinical importance of the OAR and OFR has been investigated extensively [14], [15], [17], [18]. It has been proven that the OAR and OFR are of great value in everyday practice, resulting in a reduction of x-rays. Bachmann et al [16] summarized the available literature in an excellent systematic review. The pooled sensitivity found in their study on the OAR and OFR performed by doctors was 0.96 (95% CI, 0.94-0.99), and the specificity was 0.26 (95% CI, 0.19-0.34). Evidence has also been

Limitations and future questions

Essential in assessing the accuracy of a diagnostic test (OAR and OFR) is to make use of a valid reference test. Therefore, radiography was performed for every injury. Obviously, in daily practice, the assessment of ankle sprains consists in many cases not only of clinical examination but also of radiographic assessment. We have chosen this study setting for this injury as it enables an evaluation of its sole clinical investigation. In this context, the possibility exists that, without the

Conclusion

In conclusion, regular emergency nurses are able to accurately interpret the OAR and OFR in the ED after a short, inexpensive course.

References (27)

S. Ganapathy et al.
Coping with a crowded ED: an expanded unique role for midlevel providers
Am J Emerg Med
(2003)
R.S. Hooker et al.
Emergency department uses of physician assistants and nurse practitioners: a national survey
Am J Emerg Med
(1996)
D. Debehnke et al.
The effects of a physician-nurse patient care team on patient satisfaction in an academic ED
Am J Emerg Med
(2002)
M. Sakr
Care of minor injuries by emergency nurse practitioners or junior doctors: a randomised controlled trial
Lancet
(1999)
I.G. Stiell
A study to develop clinical decision rules for the use of radiography in acute ankle injuries
Ann Emerg Med
(1992)
G.R. Auleley
Validation of the Ottawa ankle rules in France: a study in the surgical emergency department of a teaching hospital
Ann Emerg Med
(1998)
K.M. Lee
Accuracy and efficiency of X-ray requests initiated by triage nurses in an accident and emergency department
Accid Emerg Nurs
(1996)
F. Fiesseler et al.
Can nurses appropriately interpret the Ottawa Ankle Rule?
Am J Emerg Med
(2004)
M.L. Hardy et al.
Requesting and interpreting trauma radiographs: a role extension for accident and emergency nurses
Accid Emerg Nurs
(2003)
J. Allerston et al.
Nurse practitioners and the Ottawa Ankle Rules: comparisons with medical staff in requesting X-rays for ankle injured patients
Accid Emerg Nurs
(2000)

E.C. Pigman et al.

Evaluation of the Ottawa clinical decision rules for the use of radiography in acute ankle and midfoot injuries in the emergency department: an independent site assessment

Ann Emerg Med

(1994)

A. Dix

Clinical management. Where medicine meets management. Let us play

Health Serv J

(2004)

M.A. Cooper et al.

Evaluating emergency nurse practitioner services: a randomized controlled trial

J Adv Nurs

(2002)

Cited by (30)

Nurse-initiated radiographic-test protocol for ankle injuries: A randomized controlled trial
2018, International Emergency Nursing
Citation Excerpt :
The sensitivities of the OARs implemented by emergency nurses were not always 100%. In fact, previous studies found that such sensitivities ranged between 92% and 98% [23–25,12]. The result of the present study was consistent with those of Allerston and Justham [20], Lee et al. [21] and Sorensen et al. [26] who found a significant decrease in the patients’ LOS in the ED (25.0–45.0 min) when radiographic tests were initiated by emergency nurses based on the OARs at triage than when radiographic tests were requested by physicians not implementing the OARs at treatment areas.
Nurse-initiated radiographic-test protocol was compared with usual practice in reducing unnecessary ankle and foot radiographic-test requests and shortening patients’ length of stay (LOS) in an emergency department (ED) by reducing their waiting time for physician reassessment.
Patients with ankle injuries were enrolled in an unblinded randomized controlled trial. Participants were randomized to receive either the protocol (n = 56) or usual practice (n = 56). Primary outcome was the proportion of radiographic-test requests. Secondary outcomes were the proportion of fractures detected and patients’ LOS and waiting times.
The proportions of ankle and foot radiographic tests requested by triage nurses implementing the Ottawa Ankle Rules (OARs) in protocol group were smaller than those requested by physicians using their expertise in usual practice group. The proportions of malleolar and midfoot fractures detected by triage nurses implementing the OARs in protocol group were higher than those detected by physicians using their expertise in usual practice group. Patients’ LOS and waiting time from consultation to discharge in protocol group were shorter than those in usual practice group.
Implementing the nurse-initiated radiographic-test protocol reduced unnecessary ankle and foot radiographic-test requests and shortened patients’ LOS in the ED by reducing their waiting time for physician reassessment.
Diagnostic Accuracy of Clinical Decision Rules to Exclude Fractures in Acute Ankle Injuries: Systematic Review and Meta-analysis
2017, Journal of Emergency Medicine
Citation Excerpt :
The Midfoot Zone Algorithm was developed by Dayan et al. (2004) to identify clinically significant fractures of the midfoot in children with acute ankle injuries (7). The sensitivity, specificity, LR+, LR−, and DOR of the included studies are depicted in Table 3 (5–7,12,13,24–36). A meta-analysis was performed on studies investigating diagnostic accuracy of the OAR, Bernese Ankle Rules, and the Malleolar Zone Algorithm, as the number of studies evaluating each is larger than or equal to three.
Ankle decision rules are developed to expedite patient care and reduce the number of radiographs of the ankle and foot. Currently, only three systematic reviews have been conducted on the accuracy of the Ottawa Ankle and Foot Rules (OAFR) in adults and children. However, no systematic review has been performed to determine the most accurate ankle decision rule.
The purpose of this study is to examine which clinical decision rules are the most accurate for excluding ankle fracture after acute ankle trauma.
A systematic search was conducted in the databases PubMed, CINAHL, PEDro, ScienceDirect, and EMBASE. The sensitivity, specificity, likelihood ratios, and diagnostic odds ratio of the included studies were calculated. A meta-analysis was conducted if the accuracy of a decision rule was available from at least three different experimental studies.
Eighteen studies satisfied the inclusion criteria. These included six ankle decision rules, specifically, the Ottawa Ankle Rules, Tuning Fork Test, Low Risk Ankle Rule, Malleolar and Midfoot Zone Algorithms, and the Bernese Ankle Rules. Meta-analysis of the Ottawa Ankle Rules (OAR), OAFR, Bernese Ankle Rules, and the Malleolar Zone Algorithm resulted in a negative likelihood ratio of 0.12, 0.14, 0.39, and 0.23, respectively.
The OAR and OAFR are the most accurate decision rules for excluding fractures in the event of an acute ankle injury.
Effectiveness of emergency nurses’ use of the Ottawa Ankle Rules to initiate radiographic tests on improving healthcare outcomes for patients with ankle injuries: A systematic review
2016, International Journal of Nursing Studies
The Ottawa Ankle Rules provide guidelines for clinicians on the recommendation of radiographic tests to verify fractures in patients with ankle injuries. The use of the Ottawa Ankle Rules by emergency nurses has been suggested to minimise unnecessary radiographic-test requests and reduce patients’ length of stay in emergency departments. However, the findings of studies in this area are inconsistent.
A systematic review was conducted to synthesise the most accurate evidence available on the extent to which emergency nurses’ use of the Ottawa Ankle Rules to initiate radiographic tests improves healthcare outcomes for patients with ankle injuries.
The systematic review attempted to identify all relevant published and unpublished studies in English and Chinese from databases such as Ovid MEDLINE, EMBASE, ProQuest Health and Medical Complete, EBM Reviews, SPORTDiscus, CINAHL Plus, the British Nursing Index, Scopus, the Chinese Biomedical Literature Database, China Journal Net, WanFang Data, the National Central Library Periodical Literature System, HyRead, the Digital Dissertation Consortium, MedNar and Google Scholar.
Two reviewers independently assessed the eligibility of all of the studies identified during the search, based on their titles and abstracts. If a study met the criteria for inclusion, or inconclusive information was available in its title and abstract, the full text was retrieved for further analysis. The methodological quality of all of the eligible studies was assessed independently by the two reviewers.
The search of databases and other sources yielded 1603 records. The eligibility of 17 full-text articles was assessed, and nine studies met the inclusion criteria. All nine studies were subjected to narrative analysis, and five were meta-analysed. All of the studies investigated the use of the refined Ottawa Ankle Rules. The results indicated that emergency nurses’ use of the refined Ottawa Ankle Rules minimised unnecessary radiographic-test requests and reduced patients’ length of stay in emergency departments. However, the use of these rules in urgent-care departments did not reduce unnecessary radiographic-test requests or patients’ length of stay. The implementation of the refined Ottawa Ankle Rules by emergency nurses with different backgrounds, including nurse practitioners or general emergency nurses was found to reduce patients’ length of stay in emergency departments.
The results of the systematic review suggested that a nurse-initiated radiographic test protocol should be introduced as standard practice in emergency departments.
Can emergency nurses safely and accurately remove cervical spine collars in low risk adult trauma patients: An integrative review
2016, Australasian Emergency Nursing Journal
Citation Excerpt :
Accuracy is defined as free from error or defect, consistent with standard rule or model, reproducible and measurable.27 Although the concept of having a nurse clear the c-spine was first proposed in 199236 most commonly studies have appraised nurses in terms of accuracy in implementation of ankle and knee rules.41,42 This practice aids patient care, improves patient satisfaction and increases the consistency of care and improved teamwork.39,42,43
Well validated clinical decision rules exist to facilitate the safe removal of collars in the alert, orientated, low risk adult trauma patient, however this practice is traditionally conducted by medical staff. The aim of this review is to synthesise current evidence to determine the efficacy of emergency nurses in safely and accurately removing cervical spine collars using cervical spine rules, in alert, orientated, low risk trauma adult patients.
A multi-method search strategy was used to find primary research studies followed by a rigorous screening and quality appraisal process. Data from included articles were extracted, grouped and synthesised.
Nine quantitative research articles resulted in four key findings: the inter-rater reliability between nurses and doctors clearing the cervical spine was high (kappa range (0.61–0.80)); nurses can safely implement the cervical spine clinical decision rule; use of a cervical spine clinical decision rule decreases the time patients are immobilised and; nurses felt confident applying a cervical spine clinical decision rule.
Appropriately trained emergency nurses can safely apply cervical spine rules to alert, orientated, low risk adult trauma patients. Implementation of nurses clearing cervical spines should include training and ongoing monitoring.
Diagnostic performance of the Bernese versus Ottawa ankle rules: Results of a randomised controlled trial
2015, Injury
Citation Excerpt :
In many hospitals around the globe, the triage nurses order radiographs to speed up the clinical assessment process. Obviously, it is of importance to ascertain whether the triage nurses can appropriately assess the BAR as was performed earlier with the OAR [13–16]. A validation of the BAR interpreted by the triage nurses has, to our knowledge, not been performed to date.
The Ottawa ankle rules (OAR) brought about a reduction of radiographs on the Emergency Department (ED). However, still 50% of patients with ankle injuries undergo unnecessary radiography. Compared to the OAR, the Bernese ankle rule (BAR) has an acclaimed 84% reduction in radiography without loss of sensitivity. The primary aim of this study was to compare the diagnostic accuracy and reproducibility of both rules. Furthermore, the ability of triage nurses to accurately interpret the BAR was assessed.
Participants were assessed by both the ED resident and the triage nurse, applying the OAR and the BAR. After standardised data collection, ankle and foot radiographs were performed in all patients. Sensitivity and specificity of both tests applied by both observers were obtained and compared by McNemar's test. Reproducibility was calculated with Cohen's kappa.
A total of 203 patients with ankle trauma were included. For the OAR obtained by the ED residents, the sensitivity and specificity were 0.97 and 0.29, respectively. For the BAR, the sensitivity and specificity of the ED residents were 0.69 and 0.45, respectively. For the triage nurses, the OAR sensitivity and specificity were 0.86 and 0.25, respectively. The BAR sensitivity and specificity for the nurses were 0.86 and 0.40, respectively. The reproducibility of the OAR was 0.45, and for the BAR, it was 0.48.
Both rules showed comparable reproducibility. Although the BAR showed a superior specificity compared to the OAR, its sensitivity was too low to promote clinical use. The triage nurses demonstrated too low sensitivity on both rules to allow safe application. Therefore, the OAR remain the decision rules of choice for ankle injuries despite its modest ‘ruling out’ capacity.
ACR appropriateness criteria acute trauma to the ankle
2015, Journal of the American College of Radiology
Acute ankle injuries are frequently diagnosed and treated in emergency departments and outpatient clinics. Recent evidence-based clinical treatment guidelines and systematic review of economic analyses support the use of 3-view (anteroposterior, lateral, and mortise) radiographic evaluation of patients meeting the criteria of the Ottawa ankle rules. Cross-sectional imaging has a limited secondary role primarily as a tool for preoperative planning and as a problem-solving technique in patients with persistent symptoms and suspected of having occult fractures.
The ACR Appropriateness Criteria are evidence-based guidelines for specific clinical conditions that are reviewed every 3 years by a multidisciplinary expert panel. The guideline development and review include an extensive analysis of current medical literature from peer-reviewed journals and the application of a well-established consensus methodology (modified Delphi) to rate the appropriateness of imaging and treatment procedures by the panel. In those instances in which evidence is lacking or not definitive, expert opinion may be used to recommend imaging or treatment.

View all citing articles on Scopus

View full text

Original ContributionDiagnostic accuracy and reproducibility in the interpretation of Ottawa ankle and foot rules by specialized emergency nurses

Abstract

Objectives

Methods

Results

Conclusion

Introduction

Section snippets

Study design

Results

Discussion

Limitations and future questions

Conclusion

Am J Emerg Med

Am J Emerg Med

Am J Emerg Med

Lancet

Ann Emerg Med

Ann Emerg Med

Accid Emerg Nurs

Am J Emerg Med

Accid Emerg Nurs

Accid Emerg Nurs

Ann Emerg Med

Clinical management. Where medicine meets management. Let us play

Health Serv J

Evaluating emergency nurse practitioner services: a randomized controlled trial

J Adv Nurs

Original Contribution
Diagnostic accuracy and reproducibility in the interpretation of Ottawa ankle and foot rules by specialized emergency nurses