Clinical characteristics associated with adverse events in patients with exacerbation of chronic obstructive pulmonary disease: a prospective cohort study
==========================================================================================================================================================

* Ian G. Stiell
* Catherine M. Clement
* Shawn D. Aaron
* Brian H. Rowe
* Jeffrey J. Perry
* Robert J. Brison
* Lisa A. Calder
* Eddy Lang
* Bjug Borgundvaag
* Alan J. Forster
* George A. Wells

## Abstract

**Background:** To assist physicians with difficult decisions about hospital admission for patients with acute exacerbation of chronic obstructive pulmonary disease (COPD) presenting in the emergency department, we sought to identify clinical characteristics associated with serious adverse events.

**Methods:** We conducted this prospective cohort study in 6 large Canadian academic emergency departments. Patients were assessed for standardized clinical variables and then followed for serious adverse events, defined as death, intubation, admission to a monitored unit or new visit to the emergency department requiring admission.

**Results:** We enrolled 945 patients, of whom 354 (37.5%) were admitted to hospital. Of 74 (7.8%) patients with a subsequent serious adverse event, 36 (49%) had not been admitted after the initial emergency visit. Multivariable modelling identified 5 variables that were independently associated with adverse events: prior intubation, initial heart rate ≥ 110/minute, being too ill to do a walk test, hemoglobin < 100 g/L and urea ≥ 12 mmol/L. A preliminary risk scale incorporating these and 5 other clinical variables produced risk categories ranging from 2.2% for a score of 0 to 91.4% for a score of 10. Using a risk score of 2 or higher as a threshold for admission would capture all patients with a predicted risk of adverse events of 7.2% or higher, while only slightly increasing admission rates, from 37.5% to 43.2%.

**Interpretation:** In Canada, many patients with COPD suffer a serious adverse event or death after being discharged home from the emergency department. We identified high-risk characteristics and developed a preliminary risk scale that, once validated, could be used to stratify the likelihood of poor outcomes and to enable rational and safe admission decisions.

Chronic obstructive pulmonary disease (COPD), a respiratory disorder caused largely by smoking and characterized by progressive, incompletely reversible airflow obstruction, is a leading cause of hospital admission among older people. Patients who experience frequent exacerbations of COPD are at higher risk of death.1 Return to the emergency department within 30 days because of worsening respiratory symptoms was reported for 35% of COPD patients discharged from Canadian academic emergency departments.2

An important challenge facing physicians when treating patients with COPD exacerbation is deciding who should be admitted. Many of these patients will have a response to therapy in the emergency department and will not benefit from admission to hospital. A small but important number of patients have serious adverse events after hospital admission, such as death, mechanical ventilation or myocardial infarction. Others are discharged after prolonged management in the emergency department, only to experience a serious adverse event or return later to be admitted. These outcomes are important because many jurisdictions have a shortage of hospital beds and many emergency departments are overcrowded. There is, however, little evidence about risk factors for adverse events in patients with COPD to aid with disposition decisions in the emergency department, and existing guidelines are consensus based and have not been validated.3–5

The overall goal of this study was to evaluate patients with acute exacerbation of COPD seen in the emergency department to determine the clinical characteristics associated with short-term serious adverse events. Once validated, this information should help in efforts to improve and standardize admission practices for patients with COPD seeen in the emergency department, diminishing both unnecessary admissions and unsafe discharge decisions.

## Methods

### Design and setting

We conducted a prospective observational cohort study in 6 Canadian teaching hospitals in Ottawa, Ontario (2 sites); Toronto, Ont.; Kingston, Ont.; Montréal, Quebec; and Edmonton, Alberta. The combined annual emergency department volume for these hospitals was about 350 000 patient visits.6

### Study population

We included a convenience sample of adults 50 years of age or older who presented during weekday hours to the emergency department because of symptoms of acute shortness of breath secondary to exacerbation of COPD. Exacerbation of COPD was defined as an increase in at least 2 of 3 specified symptoms (breathlessness, sputum volume, sputum purulence) requiring an urgent visit to the emergency department for additional treatment.2 For all included patients, COPD had been diagnosed previously or was diagnosed during the index emergency department visit on the basis of 1-year history of chronic dyspnea or cough with sputum production. Patients must have had a history of 15 pack-years or more of cigarette smoking and prior or current evidence of moderate airflow obstruction.5

We excluded patients who were obviously too ill to be considered for discharge or who were otherwise unsuitable for the study because of resting oxygen saturation < 85%; heart rate ≥ 130/minute; systolic blood pressure < 85 mm Hg; confusion, disorientation or severe dementia; ischemic chest pain requiring treatment on arrival; acute ST elevation by electrocardiography on arrival; death from chronic illness expected within weeks; arrival from a nursing home or chronic care facility; or enrolment in the study within previous 2 months.

The research ethics boards of all 6 hospitals provided approval for this study. The research ethics boards of 3 of the hospitals specified that patients’ written informed consent was required, whereas those at the other 3 sites waived the need for written consent.

### Data collection

The patients were assessed by trained registered respiratory therapists or registered nurses. The target assessment period was 4 to 8 hours after initial treatment, but patients could be considered for enrolment from 2 to 15 hours after treatment. A central study nurse coordinator regularly evaluated the quality of the patient assessments.

We selected the variables to be assessed in the study on the basis of our clinical experience and reports in the literature.2,7–9 We collected data from each patient’s history, general examination, laboratory tests and a standardized 3-minute walk test, during which the patient walked at his or her own pace for 3 minutes, regardless of distance.10–13 Patients could use their normal walking aids but could not be supported by another person. During this test, patients used no supplementary oxygen or used their usual home oxygen flow level. The same model of recording pulse oximeter (Criticare 504DXP) was used at all sites to record heart rate and oxygen saturation levels.

### Outcome measures

The primary outcome was a serious adverse event, defined as either death from any cause within 30 days of the index emergency department visit or any of the following events within 14 days of the index emergency department visit, regardless of whether the patient was initially admitted: admission to a monitored unit, excluding monitoring by telemetry; endotracheal intubation or need for noninvasive ventilation after hospital admission, unless already receiving non-invasive ventilation at home; myocardial infarction, as defined by international consensus standards;14 major procedure (coronary artery bypass graft, percutaneous coronary intervention, other cardiac surgery or new hemodialysis); or, for patients who were discharged after the initial visit, return to the emergency department for any related medical problem within 14 days followed by admission to hospital (return to the emergency department without associated admission was not considered an adverse event). We used hospital records and provincial death records to assess the primary outcome measure, with blinding as to patient status for the predictor variables.

### Data analyses

We used appropriate univariable analyses to assess associations between the primary outcome (a serious adverse event) and clinical variables. We categorized continuous variables using the most discriminative cut points. We conducted logistic regression with stepwise selection for those variables found to be associated with serious adverse events on univariable analysis (*p* < 0.05), as well as for clinically sensible interaction terms. We conducted the analyses by visit rather than by individual patient. Using accepted approaches, we created a risk scale by rounding up the lowest logistic regression β coefficient to 1, which then served as the lowest common denominator for assigning point values to the score items.15 We assessed the classification performance of the risk score by internal validation using the bootstrap method,16,17 whereby we drew 1000 resamples with replacement from the original sample. We applied the COPD risk scale and calculated the classification performance measures (sensitivity, specificity, positive likelihood ratio and negative likelihood ratio) for each of the 1000 replicate samples. We evaluated the optimism of the bootstrap samples by comparing the averages of performance measures taken over 1000 replicates and the performance measures calculated in the original sample.

## Results

### Study sample

Of 1993 patient visits between September 2007 and April 2010, 945 met the inclusion criteria and were included in the analysis (Table 1). The other 1048 visits were excluded primarily because the patients presented when research staff were not available. The characteristics of excluded patient visits were similar (Appendix 1, available at [www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.130968/-/DC1](http://www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.130968/-/DC1)). Among the enrolled patients, there were 74 (7.8%) serious adverse events. Of concern, 36 (49%) of these occurred in the 591 patients who were not admitted at the time of the initial visit (Table 2). In addition, 2 (22%) of the 9 deaths occurred within 30 days among patients initially discharged home from the emergency department.

View this table:
[Table 1:](http://www.cmaj.ca/content/186/6/E193/T1)

Table 1: 
Characteristics of patients with exacerbation of chronic obstructive pulmonary disease seen in the emergency department

View this table:
[Table 2:](http://www.cmaj.ca/content/186/6/E193/T2)

Table 2: 
Outcomes for patients with exacerbation of chronic obstructive pulmonary disease seen in the emergency department

### Univariable and multivariable data analyses

Tables 3 and 4 show the association between a serious adverse event and variables from the history, physical examination and laboratory investigations. Some continuous variables were further categorized using cut points (Table 5).

View this table:
[Table 3:](http://www.cmaj.ca/content/186/6/E193/T3)

Table 3: 
Univariable correlation with serious adverse events for variables from history for 945 patients with chronic obstructive pulmonary disease

View this table:
[Table 4:](http://www.cmaj.ca/content/186/6/E193/T4)

Table 4: 
Univariable correlation with serious adverse events for variables from physical examination and investigations for 945 patients with chronic obstructive pulmonary disease

View this table:
[Table 5:](http://www.cmaj.ca/content/186/6/E193/T5)

Table 5: 
Univariable correlation with serious adverse events for continuous variables with discriminative cut points

The multivariable logistic regression model was developed from a reduced dataset of the 844 cases (89.3%) without missing values (Table 6). The Hosmer–Lemeshow χ2 goodness-of-fit statistic was nonsignificant (*p* = 0.7), which indicated a good fit to the data. The area under the receiver operating characteristic curve was 0.80 (95% confidence interval 0.74–0.85), meaning that the model correctly classified serious adverse events for 80% of cases. Variables that were not significant in the final model included age, walk test performance, disposition status (admitted or discharged) and several interaction terms. We conducted a secondary analysis using multiple imputation with 50 samples, and the resultant model had the same 10 variables, with only minor changes to the scores of 2 variables (Appendices 2 and 3, available at [www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.130968/-/DC1](http://www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.130968/-/DC1)).

View this table:
[Table 6:](http://www.cmaj.ca/content/186/6/E193/T6)

Table 6: 
Independent predictors of a serious adverse event, as determined by stepwise logistic regression analysis (for 844 patients with chronic obstructive pulmonary disease)*

### Preliminary risk scale

The preliminary COPD risk scale (Tables 7 and 8) consisted of 10 elements from the history (coronary bypass graft, intervention for peripheral vascular disease, intubation for respiratory distress), examination (heart rate on arrival ≥ 110/min, post-treatment oxygen saturation < 90% or heart rate ≥ 120/min) and investigations (acute ischemic changes on electrocardiography, pulmonary congestion on chest radiography, hemoglobin < 100 g/L, urea ≥ 12 mmol/L, serum carbon dioxide ≥ 35 mmol/L). The risk scale had a maximum score of 16. We found that the risk of a serious adverse event ranged from 2.2%, for a score of 0, to 91.4%, for a total score of 10. There was good calibration between observed and expected probabilities of a serious adverse event up to a score of 6 (Figure 1), beyond which there was variability because of small numbers.

View this table:
[Table 7:](http://www.cmaj.ca/content/186/6/E193/T7)

Table 7: 
Clinical variables contributing to preliminary Ottawa COPD Risk Scale to identify patients with COPD seen in the emergency department who are at high risk of a serious adverse event

View this table:
[Table 8:](http://www.cmaj.ca/content/186/6/E193/T8)

Table 8: 
Risk categories for a serious adverse event in patients with acute exacerbation of COPD seen in the emergency department

![Figure 1:](http://www.cmaj.ca/https://www.cmaj.ca/content/cmaj/186/6/E193/F1.medium.gif)

[Figure 1:](http://www.cmaj.ca/content/186/6/E193/F1)

Figure 1: 
Observed versus expected probability of serious adverse event score in patients with chronic obstructive pulmonary disease. Hosmer–Lemeshow goodness-of-fit: *p* = 0.7.

We compared the classification performance and expected admission proportions for the Ottawa COPD Risk Scale with current practice (as reflected by current study data) at the 6 study hospitals (Table 9). Use of the scale could improve upon the sensitivity of current practice, which had only 51% (38/74) of patients who subsequently experienced a serious adverse event being admitted on the first emergency department visit. For example, choosing total point scores of 1, 2 or 3 as the threshold for admission would be associated with sensitivities for a serious adverse event of 91.2%, 80.9% or 60.3%, respectively. These theoretical admission thresholds would result in admission rates of 57.6%, 43.2% or 20.0%, respectively, as compared with the observed admission rate of 37.5% at the study sites.

View this table:
[Table 9:](http://www.cmaj.ca/content/186/6/E193/T9)

Table 9: 
Classification performance and expected admission rates for preliminary COPD Risk Scale scores, compared with current practice at study hospitals

The classification performance of the final score categories was validated using the sensitivity, specificity, positive likelihood ratio and negative likelihood ratio. This internal validation showed the risk scores to be highly accurate across 1000 replications with the bootstrap method (Appendix 4, available at [www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.130968/-/DC1](http://www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.130968/-/DC1)).16

## Interpretation

Among patients who presented to the emergency department with acute exacerbation of COPD, we found a relatively high frequency of serious adverse events, a modest hospital admission rate and a concerning proportion of poor outcomes among patients initially discharged home. We found that some 20 clinical and laboratory predictors were strongly associated with the development of serious adverse events. Multivariable analyses resulted in a concise and accurate model involving 10 unique, independent, high-risk factors, 4 from the initial clinical assessment, 5 from bedside investigations and 1 from reassessment after emergency department treatment. These high-risk factors are readily available and do not require sophisticated imaging or expensive testing to assist decision-making, yet have not been explicitly presented in previous models. These factors could assist physicians in identifying patients most at risk for adverse outcomes and most in need of hospital admission. The 10-element preliminary COPD risk scale provides a quantitative estimate of the risk of poor outcomes. We expect that this risk scale, once fully validated, will be widely used to improve both hospital admission practices and the safety of management decisions in the emergency department.18–20

We are concerned by the number of serious adverse events among patients with COPD who were discharged from the emergency department. Identification of high-risk characteristics by physicians has the potential to substantially improve patient safety by helping to ensure that patients who are most at risk for poor outcomes are admitted. Although Canadian hospitals would struggle to admit 80% of patients with COPD, as we understnad is currently done in US hospitals, we believe that even a modest increase in admission rates (e.g., to 50% from the current 37.5%) would very likely lead to safer management practices. More important than increasing the admission percentage is ensuring admission of the correct patients, i.e., those at greatest risk of a poor outcome. Alternatively, the scale could be used to identify at-risk patients who should have guaranteed early follow-up, perhaps in specialized COPD clinics.

There have been no large, robust evaluations of risk factors to assist with admission decisions for patients with COPD in the emergency department. Previous prospective studies conducted in the emergency department had one or more of these limitations: small sample size, no inclusion of response to therapy, no functional testing such as a walk test and no prospective validation.7,21–24 A larger study (in France) was limited by no evaluation of response to therapy and lack of follow-up mortality data.25 Other prospective studies conducted only after admission to hospital have been limited by similar concerns, and most of these attempted only to predict mortality.26–30 Others involved retrospective analyses of existing databases, were focused on inpatients or mortality, and were not directly applicable to the emergency department.31–35

### Strengths and limitations

Our study had several strengths, including multicentre and rigorous prospective collection of real-time clinical data, comprehensive follow-up and unique use of the 3-minute walk test. Nonetheless, some aspects of the study warrant discussion. First, the study included both admitted and discharged patients, which we believe is the correct methodologic approach. Admission status may confound the likelihood of a serious adverse event, since it is possible that some admitted patients will not experience an adverse event because they receive more intensive therapy in hospital. Conversely, those same patients might have experienced an adverse event if they had been discharged home. Our objectives, however, were to identify high-risk characteristics and develop a preliminary risk scale that ensures appropriate admission of high-risk patients (i.e., sensitivity) while minimizing the admission of low-risk patients (i.e., specificity). We could do this only by evaluating admitted patients. We also note that admitted status was not significantly associated with serious adverse events in the multivariable model.

We chose not to evaluate spirometry findings in the emergency department, because this modality is often unavailable and was thought to be of limited usefulness for predicting short-term outcomes. We believe that use of the “monitored unit” (not telemetry) as a criterion for a serious adverse event is an important outcome that almost always reflects severity of illness. Patients treated in a monitored unit would likely have experienced substantial morbidity if they had been discharged, because they were bedridden and required constant cardiac monitoring. This preliminary risk scale should be prospectively validated in a new set of emergency department patients with COPD. Although we had few outcomes relative to the number of predictors, internal validation with bootstrapping showed that the estimate and classification performance for risk scores were stable.36 We chose not to adjust the analysis, despite the fact that some patients were enrolled more than once. We were unable to enrol a large number of eligible patients who presented outside of normal business hours, but we detected no selection bias. Some readers may be surprised that 4 cardiovascular variables were associated with serious adverse events in patients with a respiratory condition, but cardiovascular disease and COPD are well known as common comorbidities that lead to worse outcomes than either condition alone.3,5,25,37 In addition, we know that some exacerbations of COPD are accompanied by simultaneous exacerbation of heart failure. Finally, it is possible, although unlikely, that some return emergency department visits were not identified among patients who returned to a different hospital. If such events did occur, we believe that they would not have had a significant effect on the results.

### Conclusion

Patients with exacerbation of COPD are commonly admitted to hospital, but many such patients are discharged home from the emergency department and then experience serious adverse events or death. We have identified high-risk characteristics and developed a unique risk scale that can be used to stratify the risk of poor outcomes for patients with COPD seen in the emergency department and to enable rational and safe disposition decisions. Once validated, this scale will ultimately benefit both patients and health care systems by ensuring appropriate admissions, targeting those who need early follow-up and diminishing unnecessary hospital admissions.

## Acknowledgements

The authors gratefully acknowledge the invaluable assistance of the following individuals from the study sites: Antoinette Colacone RN, Chris Tselios BSc and Yanka Mihaylova, Jewish General Hospital, Montréal, Que.; Katherine Bowes RN and Jane Reid RN, Kingston General Hospital, Kingston, Ont.; Michelle Loftus RN, Mount Sinai Hospital, Toronto, Ont.; Sara Dorrington RRT, James Acheson RRT, Michael Gatta RRT, Michelle Saulnier RRT, Krista Bangs RRT, Christine Belisle RRT and Vanessa Lawlor RRT, The Ottawa Hospital, Civic and General campuses, Ottawa, Ont.; Debbie Boyko RN and Dion Pasichnyk, University of Alberta Hospital, Edmonton, Alta. In addition, the authors thank the following individuals from the coordinating centre in Ottawa: Tami Clavet RN, Jennifer Brinkhurst, Connor Sheehan, Angela Marcantonio, Sheryl Domingo and My-Linh Tran, Ottawa Hospital Research Institute; Carol Ann Jodouin, Department of Pathology and Laboratory Medicine, The Ottawa Hospital; and Li Chen, University of Ottawa Heart Institute. Finally, the authors are grateful to the many physicians, nurses and respiratory therapists at the study hospitals for their helpful assistance with patient identification and data collection.

## Footnotes

*   **Competing interests:** For studies outside the work reported here, Brian Rowe’s institution has received grants from GlaxoSmithKline and Electrocore (investigator-initiated) and from MedImmune/AstraZeneca (industry-initiated). Brian Rowe has also received payment for lectures to general practitioners from Bristol Myers Squibb and AstraZeneca. Bjug Borgundvaag has received payment for lectures to general practitioners from Bristol Myers Squibb. No competing interests declared by other authors.

*   This article has been peer reviewed.

*   **Disclaimer:** George Wells is a statistical consultant for *CMAJ.* He was not involved in the editorial decision-making for this article.

*   **Contributors:** Ian Stiell conceived the idea, secured research funding and prepared the manuscript. Catherine Clement coordinated the study, collected data and contributed to writing the manuscript. Shawn Aaron, Brian Rowe, Jeffrey Perry, Robert Brison, Lisa Calder, Eddy Lang, Bjug Borgundvaag and Alan Forster assisted with study design, supervised the recruitment of patients and management of data, and revised the manuscript. George Wells provided considerable statistical assistance and revised the manuscript. All authors contributed to supervising the conduct of the trial and data collection and to drafting or revising the manuscript, and all approved the final version. Ian Stiell had full access to all of the study data and takes responsibility for integrity of the data and accuracy of the data analysis.

*   **Funding:** This study was supported by peer-reviewed funding from the Canadian Institutes of Health Research (MOP 82742). Ian Stiell holds a Distinguished Professorship and University Health Research Chair from the University of Ottawa. Shawn Aaron holds a Clinical Research Chair from the University of Ottawa. Brian Rowe is supported by a Tier I Canada Research Chair in Evidence-based Emergency Medicine. Jeffrey Perry holds a New Investigator Award from the Canadian Institutes of Health Research and a Clinical Research Chair from the University of Ottawa.

## References

1.  Seemungal TAR, Donaldson GC, Paul EA, et al. Effect of exacerbation on quality of life in patients with chronic obstructive pulmonary disease. Am J Respir Crit Care Med 1998;157:1418–22.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1164/ajrccm.157.5.9709032&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=0009603117&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000073570700009&link_type=ISI) 

2.  Aaron SD, Vandemheen K, Hebert P, et al. Outpatient oral prednisone after emergency treatment of chronic obstructive pulmonary disease. N Engl J Med 2003;348:2618–25.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJMoa023161&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=12826636&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000183726400005&link_type=ISI) 

3.  O’Donnell DE, Aaron S, Bourbeau J, et al. Canadian Thoracic Society recommendations for management of chronic obstructive pulmonary disease — 2007 update. Can Respir J 2007;14 Suppl B:5B–32B.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=17885691&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000254274800001&link_type=ISI) 

4.  Chronic obstructive pulmonary disease: management of chronic obstructive pulmonary disease in adults in primary and secondary care. London (UK): National Institute for Health and Clinical Excellence; 2010.
    
    

5.  Global strategy for the diagnosis, management, and prevention of chronic obstructive pulmonary disease. Global Initiative for Chronic Obstructive Lung Disease; updated 2014. Available: [www.goldcopd.org/guidelines-global-strategy-for-diagnosis-management.html](http://www.goldcopd.org/guidelines-global-strategy-for-diagnosis-management.html) (accessed 2014 Feb. 11).
    
    

6.  Stiell IG, Clement CM, Brison RJ, et al. A risk scoring system to identify emergency department patients with heart failure at high risk for serious adverse events. Acad Emerg Med 2013;20:17–26.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1111/acem.12056&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23570474&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 

7.  Kim S, Emerman CL, Cydulka RK, et al. Prospective multicenter study of relapse following emergency department treatment of COPD exacerbation. Chest 2004;125:473–81.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1378/chest.125.2.473&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=14769727&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000188978400021&link_type=ISI) 

8.  Soler JJ, Sanchez L, Roman P, et al. Risk factors of emergency care and admissions in COPD patients with high consumption of health resources. Respir Med 2004;98:318–29.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.rmed.2003.04.001&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=15072172&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000220633100007&link_type=ISI) 

9.  Roberts CM, Ryland I, Lowe D, et al. Audit of acute admissions of COPD: standards of care and management in the hospital setting. Eur Respir J 2001;17:343–9.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiZXJqIjtzOjU6InJlc2lkIjtzOjg6IjE3LzMvMzQzIjtzOjQ6ImF0b20iO3M6MjE6Ii9jbWFqLzE4Ni82L0UxOTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

10. Enright PL. The six-minute walk test. Respir Care 2003;48:783–5.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6ODoicmVzcGNhcmUiO3M6NToicmVzaWQiO3M6ODoiNDgvOC83ODMiO3M6NDoiYXRvbSI7czoyMToiL2NtYWovMTg2LzYvRTE5My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

11. Brooks D, Parson J, Tran D, et al. The two-minute walk test as a measure of functional capacity in cardiac surgery patients. Arch Phys Med Rehabil 2004;85:1525–30.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.apmr.2004.01.023&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=15375829&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000223736400024&link_type=ISI) 

12. Sciurba F, Criner GJ, Lee SM, et al. Six-minute walk distance in chronic obstructive pulmonary disease. Am J Respir Crit Care Med 2003;167:1522–7.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1164/rccm.200203-166OC&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=12615634&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000183078300014&link_type=ISI) 

13. Pan AM, Stiell IG, Clement CM, et al. Feasibility of a structured 3-minute walk test as a clinical decision tool for patients presenting to the emergency department with acute dyspnea. Emerg Med J 2009;26:278–82.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NzoiZW1lcm1lZCI7czo1OiJyZXNpZCI7czo4OiIyNi80LzI3OCI7czo0OiJhdG9tIjtzOjIxOiIvY21hai8xODYvNi9FMTkzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

14. Alpert JS, Thygesen K, Antman E, et al. Myocardial infarction redefined — a consensus document of the Joint European Society of Cardiology/American College of Cardiology Committee for the redefinition of myocardial infarction. J Am Coll Cardiol 2000; 36:959–69.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0735-1097(00)00804-4&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10987628&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000089152100048&link_type=ISI) 

15. Le Gal G, Righini M, Roy PM, et al. Prediction of pulmonary embolism in the emergency department: the revised Geneva score. Ann Intern Med 2006;144:165–71.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.7326/0003-4819-144-3-200602070-00004&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=16461960&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000235074800002&link_type=ISI) 

16. Efron B, Tibshirani R. Bootstrap methods for standard errors, confidence intervals, and other measures of statistical accuracy. Stat Sci 1986;1:54–77.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1214/ss/1177013815&link_type=DOI) 

17. Steyerberg EW, Harrell FE Jr., Borsboom GJ, et al. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001;54:774–81.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0895-4356(01)00341-9&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=11470385&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000170019900003&link_type=ISI) 

18. Stiell IG, Wells G, Laupacis A, et al. A multicentre trial to introduce clinical decision rules for the use of radiography in acute ankle injuries. BMJ 1995;311:594–7.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjEyOiIzMTEvNzAwNS81OTQiO3M6NDoiYXRvbSI7czoyMToiL2NtYWovMTg2LzYvRTE5My5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

19. Stiell IG, Clement C, McKnight RD, et al. The Canadian C-spine Rule versus the NEXUS low-risk criteria in patients with trauma. N Engl J Med 2003;349:2510–8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJMoa031375&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=14695411&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000187529500006&link_type=ISI) 

20. Stiell IG, Clement CM, Grimshaw J, et al. Implementation of the Canadian C-Spine Rule: prospective 12-centre cluster randomised trial. BMJ 2009;339:b4146.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1136/bmj.b4146&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=19875425&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 

21. Emerman CL, Connors AF, Lukens T, et al. Relationship between arterial blood gases and spirometry in acute exacerbations of chronic obstructive pulmonary disease. Ann Emerg Med 1989; 18:523–7.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0196-0644(89)80837-6&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=2497664&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1989U470400008&link_type=ISI) 

22. Emerman CL, Effron D, Lukens TW. Spirometric criteria for hospital admission of patients with acute exacerbation of COPD. Chest 1991;99:595–9.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1378/chest.99.3.595&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=1995214&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1991FA84800015&link_type=ISI) 

23. Tsai CL, Clark S, Cydulka RK, et al. Factors associated with hospital admission among emergency department patients with chronic obstructive pulmonary disease exacerbation. Acad Emerg Med 2007;14:6–14.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1197/j.aem.2006.07.034&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=17119187&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000243449200002&link_type=ISI) 

24. Rowe BH, Willis V, Mackey D, et al. Prospective multicenter study of treatment and relapse following emergency department discharge for acute COPD. Acad Emerg Med 2009;16:316–24.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1111/j.1553-2712.2009.00366.x&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=19298621&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 

25. Roche N, Zureik M, Soussan D, et al. Predictors of outcomes in COPD exacerbation cases presenting to the emergency department. Eur Respir J 2008;32:953–61.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiZXJqIjtzOjU6InJlc2lkIjtzOjg6IjMyLzQvOTUzIjtzOjQ6ImF0b20iO3M6MjE6Ii9jbWFqLzE4Ni82L0UxOTMuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

26. Mohan A, Bhatt SP, Mohan C, et al. Derivation of a prognostic equation to predict in-hospital mortality and requirement of invasive mechanical ventilation in patients with acute exacerbation of chronic obstructive pulmonary disease. Indian J Chest Dis Allied Sci 2008;50:335–42.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=19035052&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 

27. Ranieri P, Bianchetti A, Margiotta A, et al. Predictors of 6-month mortality in elderly patients with mild chronic obstructive pulmonary disease discharged from a medical ward after acute nonacidotic exacerbation. J Am Geriatr Soc 2008;56:909–13.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1111/j.1532-5415.2008.01683.x&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=18384582&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000255486000019&link_type=ISI) 

28. Chang CL, Sullivan GD, Karalus NC, et al. Predicting early mortality in acute exacerbation of chronic obstructive pulmonary disease using the CURB65 score. Respirology 2011;16:146–51.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1111/j.1440-1843.2010.01866.x&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=20920140&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 

29. Matkovic Z, Huerta A, Soler N, et al. Predictors of adverse outcome in patients hospitalised for exacerbation of chronic obstructive pulmonary disease. Respiration 2012;84:17–26.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1159/000335467&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=22327370&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000306077400005&link_type=ISI) 

30. Steer J, Gibson J, Bourke SC. The DECAF Score: predicting hospital mortality in exacerbations of chronic obstructive pulmonary disease. Thorax 2012;67:970–6.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6OToidGhvcmF4am5sIjtzOjU6InJlc2lkIjtzOjk6IjY3LzExLzk3MCI7czo0OiJhdG9tIjtzOjIxOiIvY21hai8xODYvNi9FMTkzLmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

31. Jones RC, Donaldson GC, Chavannes NH, et al. Derivation and validation of a composite index of severity in chronic obstructive pulmonary disease: the DOSE Index. Am J Respir Crit Care Med 2009;180:1189–95.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1164/rccm.200902-0271OC&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=19797160&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000272889200006&link_type=ISI) 

32. Sharma G, Kuo YF, Freeman JL, et al. Outpatient follow-up visit and 30-day emergency department visit and readmission in patients hospitalized for chronic obstructive pulmonary disease. Arch Intern Med 2010;170:1664–70.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/archinternmed.2010.345&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=20937926&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000282917600009&link_type=ISI) 

33. Asiimwe AC, Brims FJ, Andrews NP, et al. Routine laboratory tests can predict in-hospital mortality in acute exacerbations of COPD. Lung 2011;189:225–32.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1007/s00408-011-9298-z&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=21556787&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000291062600007&link_type=ISI) 

34. Sundh J, Janson C, Lisspers K, et al. The Dyspnoea, Obstruction, Smoking, Exacerbation (DOSE) index is predictive of mortality in COPD. Prim Care Respir J 2012;21:295–301.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.4104/pcrj.2012.00054&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=22786813&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 

35. Tabak YP, Sun X, Johannes RS, et al. Development and validation of a mortality risk-adjustment model for patients hospitalized for exacerbations of chronic obstructive pulmonary disease. Med Care 2013;51:597–605.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1097/MLR.0b013e3182901982&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23604015&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 

36. Vittinghoff E, McCulloch CE. Relaxing the rule of ten events per variable in logistic and Cox regression. Am J Epidemiol 2007; 165:710–8.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1093/aje/kwk052&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=17182981&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000244655200014&link_type=ISI) 

37. Scala R, Bartolucci S, Naldi M, et al. Co-morbidity and acute decompensations of COPD requiring non-invasive positive-pressure ventilation. Intensive Care Med 2004;30:1747–54.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=15258727&link_type=MED&atom=%2Fcmaj%2F186%2F6%2FE193.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000225104900009&link_type=ISI)