Abstract
Background: We previously demonstrated improvement in bronchopulmonary dysplasia and nosocomial infection among preterm infants at 12 neonatal units using the Evidence-based Practice for Improving Quality (EPIQ). In the current study, we assessed the association of Canada-wide implementation of EPIQ with mortality and morbidity among preterm infants less than 29 weeks gestational age.
Methods: This prospective cohort study included 6026 infants admitted to 25 Canadian units between 2008 and 2012 (baseline year, n = 1422; year 1, n = 1611; year 2, n = 1508; year 3, n = 1485). Following a 1-year baseline period and 6 months of training and planning, EPIQ was implemented over 3 years. Our primary outcome was a composite of neonatal mortality and any of bronchopulmonary dysplasia, severe neurologic injury, severe retinopathy of prematurity, necrotizing enterocolitis and nosocomial infection. We compared outcomes for baseline and year 3 using multivariable analyses.
Results: In adjusted analyses comparing baseline with year 3, the composite outcome (70% v. 65%; adjusted odds ratio [OR] 0.63, 95% confidence interval [CI] 0.51 to 0.79), severe retinopathy (17% v. 13%; OR 0.60, 95% CI 0.45 to 0.79), necrotizing enterocolitis (10% v. 8%; OR 0.73, 95% CI 0.52 to 0.98) and nosocomial infections (32% v. 24%; OR 0.63, 95% CI 0.48 to 0.82) were significantly reduced. The composite outcome was lower among infants born at 26 to 28 weeks gestation (62% v. 52%; OR 0.62, 95% CI 0.49 to 0.78) but not among infants born at less than 26 weeks gestational age (90% v. 88%; OR 0.73, 95% CI 0.44 to 1.20).
Interpretation: EPIQ methodology was generalizable within Canada and was associated with significantly lower likelihood of the composite outcome, severe retinopathy, necrotizing enterocolitis and nosocomial infections. Infants born at 26 to 28 weeks gestational age benefited the most.
Rates of neonatal mortality and morbidity have not improved substantially over the past decade, despite advances in neonatal and perinatal care.1–3 To improve infant outcomes, quality improvement initiatives have been implemented in neonatal units, but their success has varied.4–6 A common approach is to form collaborative neonatal networks and to use quality improvement methods to address variations in practice and outcomes.7–9 Commonly used quality improvement methods include identifying the best available evidence, adapting practice change to the organization, collaborative learning, and audit and feedback of information implemented in iterative “plan, do, study, act” cycles.10
We previously developed and tested the Evidence-based Practice for Improving Quality (EPIQ) in 12 neonatal units in a cluster randomized trial.11 EPIQ is a multifaceted quality improvement approach that combines the best available evidence with institution-specific data to identify institution-specific needs. Sites targeting nosocomial infection achieved a 32% reduction in infections, and sites targeting bronchopulmonary dysplasia achieved a 15% reduction in that condition. Five units that did not participate formed a comparison group and showed no improvement for either outcome.11
In the current study, our objective was to assess the generalizability of EPIQ to other neonatal units in Canada and to determine any association with improvements in mortality and morbidity among preterm neonates born at less than 29 weeks gestational age.
Methods
This prospective cohort study involved 25 of the 30 tertiary-level neonatal units within Canada that participate in the Canadian Neonatal Network and included 95% of all eligible neonatal admissions to those units. Of the remaining units, 3 were small, with insufficient resources to implement EPIQ, and 2 were not fully accredited as tertiary-level units at the time of study commencement. We included all infants less than 29 weeks gestational age who were admitted to the participating units, except those for whom palliative care was planned at birth. The 25 participating sites had a median of 26 (interquartile range [IQR] 14–36) beds and admitted a median of 69 (IQR 31–128) neonates per year during the period 2008 to 2012. The 12 sites from the initial EPIQ trial were included in this project. Infants from the 13 additional sites had higher mean birth weight (971 v. 924 g), and a lower proportion had an Apgar score below 7 (36% v. 42%). We calculated gestational age using a hierarchical algorithm ordered as date of in vitro fertilization, early ultrasound, last menstrual period, obstetric estimate and pediatric estimate.
Conducted from April 2008 to September 2012, the study consisted of a 1-year period for collecting baseline data, a 6-month quality improvement training and planning period, and a 3-year intervention period. Participating units chose the outcomes to be targeted and the practice changes to be implemented. All units were aware of the others’ activities via a web portal.12 The study was approved by the institutions’ ethics review boards. Informed consent from patients was waived, as the activities were considered part of routine quality improvement. The data monitoring committee recommended continuation of the study on an annual basis.
Outcomes
The primary outcome was a composite of death or any of 5 major morbidities: bronchopulmonary dysplasia (need for supplemental oxygen at 36 weeks corrected gestational age or time of transfer to a level-2 facility13), severe neurologic injury (grade ≥ 3 intraventricular hemorrhage14 or intraparenchymal echogenicity or periventricular leukomalacia), severe retinopathy of prematurity (stage ≥ 215 or need for treatment in either eye), necrotizing enterocolitis (stage ≥ 216) and nosocomial infection (positive results on culture of blood or cerebrospinal fluid samples obtained after 2 days of age from a symptomatic neonate17). We also examined these outcomes independently as secondary outcomes. The oxygen reduction test was not used for diagnosing bronchopulmonary dysplasia. The numbers of infants who underwent neurologic imaging or an eye examination were less than the total number of infants included in the study. All data were abstracted from patient charts according to standardized definitions18 and were transmitted electronically to the Coordinating Centre.
Baseline data collection, quality improvement training and planning
During the baseline period, we conducted interviews, focus groups, group sessions and staff surveys at each unit to evaluate organizational culture and behaviour. We reported results back to the units, along with our recommendations. Site investigators established EPIQ teams, which typically included a neonatologist, nurse manager, nurse educator, quality-improvement personnel and outcome-specific personnel. Site investigators and 2 or 3 team members from each unit attended training and a planning workshop.
Training covered use of Pareto charts, histograms, control charts, trend analysis, rapid-cycle improvement models19 and change management techniques. We emphasized the value of identifying nonambiguous clinical aims, team-building and leadership, defragmenting the process to identify barriers, developing flow charts of existing and “ideal” processes, and generating feedback. Participants then trained their hospital teams. The planning workshop consisted of group sessions to identify best practices. Each group created lists of interventions that were known or anticipated to improve a specific outcome and then performed systematic reviews using a rapid review format. Unit teams evaluated their own baseline data, selected outcomes to target and developed process measures.
Information on proposed interventions, plans, dissemination materials and process measures were shared among the teams. Toward the end of the baseline training period, outcome groups reviewed the accumulated evidence, data analysis, clinical process analyses and qualitative study reports. We classified the interventions into 3 categories according to the evidence (definitive, nondefinitive and unsubstantiated) and established consensus recommendations. Evidence and recommendations were continually reviewed throughout the study. Training and planning workshops were repeated annually.
Interventions
Participating units implemented practice changes every 3 to 6 months using data from process indicators, quarterly benchmarking reports and consensus recommendations, according to local organizational and cultural issues. We monitored compliance at random intervals. At the annual workshops, each site reported its successes, failures and barriers to implementing change. Top-performing sites for each outcome shared their approaches. Discussion and opportunities for mutual learning were facilitated. We provided quarterly feedback to the sites regarding their outcomes and process performance in comparison with the network. Midway through the intervention period, site representatives were encouraged to visit other sites to observe practices and share change strategies.
Statistical analyses
We made an a priori decision to compare the change in rate of the composite outcome between baseline and year 3 of the intervention period (for the power calculation, see Appendix 2, available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.140399/-/DC1). We summarized characteristics of the study population using descriptive statistics. We compared infant characteristics among the baseline and intervention years using the χ2 test for categorical measures and the F test or Kruskal–Wallis test, as appropriate, for continuous measures. We examined homogeneity of the intervention effects across hospitals using a forest plot of hospital-specific odds ratios (ORs) with a summary OR estimated using a random-effects model. When significant change in an adverse outcome was observed, we estimated number needed to treat and 95% confidence intervals (CIs) from raw outcome rates without adjustment. To account for site effects and the heterogeneity of intervention effects across sites, we applied hierarchical logistic regression models20 with random intercept and random site-intervention interaction adjusting for potential confounders (gestational age, small for gestational age, Apgar score < 7 at 5 minutes, Score for Neonatal Acute Physiology version II21 > 20) to examine the change in rate of outcome from baseline to each intervention year. We used the compound symmetric covariance structure for the regressions. We performed stratified analyses for groups with gestational age less than 26 weeks and gestational age 26 to 28 weeks. For the entire cohort and the subgroups with gestational age less than 26 weeks and 26 to 28 weeks, gestational age was used as a continuous variable.
We conducted exploratory analyses to assess the effects of specific interventions and their association with outcome from sites that reported implementing each practice change. For each intervention, we compared pre- and post-implementation data using unadjusted analyses. We used SAS version 9.3 and R version 2.15.1 software for data management and statistical analyses. We applied a significance level of 0.05 for 2-sided tests, without multiple-comparison adjustment, as these analyses were complementary to the primary planned analyses.
Results
The 25 sites implemented a total of 130 practice changes targeting a specific outcome and 30 interventions that improved the care process. The number of practice changes implemented per site ranged from 2 to 8 and were grouped by outcome (Table 1 and Appendix 2); however, some changes could affect multiple outcomes. The most commonly adopted changes were use of a T-piece resuscitator, early and adequate use of continuous positive airway pressure, controlled oxygen use, and early use of surfactant and/or early extubation for bronchopulmonary dysplasia; delayed cord clamping for neurologic injury; controlled oxygen use for retinopathy of prematurity; standardization of feeding guidelines and donor milk for necrotizing enterocolitis; and hand hygiene and central line management practices for nosocomial infection.
Practice changes implemented by participating sites during the intervention years
The proportions of infants admitted to units at less than 26 weeks gestation, with Apgar score less than 7 at 5 minutes, and with Score for Neonatal Acute Physiology version II above 20 were greater in year 3 than in the baseline year (Table 2). Process measures confirmed implementation of various practice changes (e.g., decreased use of mechanical ventilation, older age at first administration of surfactant, increased use of magnesium sulphate).
Infant and maternal characteristics and process measures
Compared with baseline, there was a reduction in the composite outcome in year 3, as well as in severe retinopathy and nosocomial infections (Table 3). Mild heterogeneity (I2 = 44%) was observed among site-specific ORs for the composite outcome (Figure 1). The summary OR, estimated using a random-effects model, was 0.74 (95% CI 0.59 to 0.94). After multivariable analyses, we observed statistically significant reductions in year 3 compared with baseline for the composite outcome (OR 0.63, 95% CI 0.51 to 0.79; number needed to treat to prevent 1 incident [NNT] 18, 95% CI 12 to 45), severe retinopathy (OR 0.60, 95% CI 0.45 to 0.79; NNT 26, 95% CI 14 to 133), necrotizing enterocolitis (OR 0.73, 95% CI 0.52 to 0.98; NNT 50, 95% CI 25 to 714) and nosocomial infection (OR 0.63, 95% CI 0.48 to 0.82; NNT 14, 95% CI 10 to 25) (Table 4). No statistically significant differences in mortality (OR 0.86, 95% CI 0.65 to 1.13), bronchopulmonary dysplasia (OR 0.81, 95% CI 0.64 to 1.01) or severe neurologic injury (OR 0.91, 95% CI 0.69 to 1.19) were observed. The majority of improvements were observed in infants born at 26 to 28 weeks gestational age. Improvement in outcomes accelerated as the study progressed, with maximum benefit in year 3 (Table 4 and Appendix 2).
Neonatal outcomes during study period
Forest plot of odds ratios (ORs) for the composite outcome between baseline and intervention year 3 by site (identifed by letters). The summary OR was calculated using a random-effects model. Statistical heterogeneity (I2) was 44%. CI = confidence interval.
Adjusted comparison of neonatal outcomes between baseline and 3 intervention years
Table 5 shows the association of specific practice changes with outcomes using site-specific analyses. Three practice changes were identified as significant contributors to outcomes change: early use of surfactant and/or early extubation was associated with a decrease of 17.7 (95% CI 10.2 to 25.0) percentage points in bronchopulmonary dysplasia, use of colostrum or enhanced breast milk with a decrease of 5.3 (95% CI 1.9 to 8.9) percentage points in necrotizing enterocolitis and central line management with decreases of 9.3 (single intervention, 95% CI 5.5 to 12.9) and 5.7 (bundled intervention, 95% CI 2.8 to 8.6) percentage points in nosocomial infections. Delayed cord clamping was reported as a practice change by 7 sites; however, of 521 infants in the postimplementation phase, only 166 actually received delayed cord clamping, and of these 22% had severe neurologic injury (same rate as entire cohort).
Association of practice changes with neonatal outcomes, according to reported practice changes
Interpretation
In this prospective cohort study, embracing the entire EPIQ process was likely key to successful change management and more important than any single intervention. Certain specific interventions were identified as contributing significantly to improvement in outcomes, but the study was underpowered to infer these associations as conclusive. Our finding that targeting all important neonatal outcomes led to success in some outcomes and little change in others was unsurprising, given that units were free to identify their own targets. The lack of a reduction in mortality may have been due to residual confounding related to higher prematurity and higher illness severity during the later intervention years. In year 3, we observed a reduction in the rate of bronchopulmonary dysplasia; however, continued monitoring is required to determine the sustainability of this change. Delaying the age at which surfactant was administered without a significant reduction in the number of infants requiring surfactant suggests that proper evaluation of change in strategies is warranted.
Although our initial EPIQ trial11 reported significant reductions in infections and bronchopulmonary dysplasia, we were not able to replicate those results when the trial interventions were offered to all other Canadian units as best practice guidelines.22 In the later study, continued improvement was observed in the 11 units from the initial EPIQ trial but not the 14 units that were new to EPIQ. This result is consistent with previous observations that practice guidelines do not necessarily result in improvement23 and that multifaceted programs aimed at changing behaviours are more effective for quality improvement.24 Thus, a collaborative decision was made to involve all units in this third study of EPIQ. Our approach allowed us to test the generalizability of EPIQ in a more pragmatic setting, as each unit had different needs, some interventions affected multiple outcomes, and many units routinely targeted multiple outcomes simultaneously. The downside of our approach was that many units targeted outcomes such as nosocomial infections and retinopathy and made little effort to address other outcomes.
Our findings are consistent with reports from other neonatal quality improvement initiatives, such as the Vermont Oxford Network.25 Although their approach differed from ours in some respects, both were based on evidence review, cultivating quality improvement at a systems level and using team approaches to implement practice changes. The Vermont Oxford Network has reported decreased length of stay and improvements in infant growth,26,27 severe retinopathy, neurologic injury, oxygen use at discharge28 and infections.29 Similarly, the California Perinatal Care Collaborative showed decreased rates of necrotizing enterocolitis,30 increased administration of antenatal steroids31 and decreased infections following state-based quality improvement efforts.6 The difference between these approaches and ours lies in our use of central coordination and continued support; our ability to address multiple interventions for multiple outcomes; unit-led practice changes in our study, rather than central enforcement, which created a culture of autonomy, accountability and responsibility; and annual in-person meetings and training that fostered extensive collaboration.
Limitations
Our study involved many different interventions, and was not powered to identify the individual effects of a specific intervention. We conducted exploratory analyses to address this issue but acknowledge that the current report involves multiple comparisons without statistical adjustment and that the observed improvements may be attributable to overall culture change rather than specific interventions. It is also possible that some interventions took longer than planned to implement, as for delayed cord clamping. In addition, similar to the situation for other observational studies, we were unable to ascertain the effects of secular change, other uncontrolled interventions and changes in management practices on the observed differences in outcomes. Variations in outcomes by site were evident in each quarterly report, and these variations should be explored, along with interactions among multiple interventions. The number of smaller, younger and sicker infants increased as the study progressed, which emphasizes the importance of stratified and adjusted analyses and suggests possible residual confounding related to prematurity and illness severity. However, we observed incremental improvement in outcomes despite increased severity of illness on admission, which suggests that quality improvement initiatives take time to produce results. Variation in the number of interventions by individual units may also have limited the generation of specific intervention–time–outcome relationships. However, this pragmatic approach improves the generalizability of findings.
Conclusion
In this study, we have shown that EPIQ is generalizable within Canada and is associated with a significantly lower likelihood of a composite adverse outcome, severe retinopathy, necrotizing enterocolitis and nosocomial infections among infants born at less than 29 weeks gestational age, with those born at 26 to 28 weeks gestational age benefiting the most. Further research is needed to address outcomes in those born at less than 26 weeks gestational age. Further work to develop guided practice changes, perform accountability audits and generate a higher level of collaboration through mutual site visits is underway, with the launch of a new project in February 2014 that includes all 30 Canadian units.
Acknowledgements
The authors would like to thank the staff of the EPIQ Study Coordinating Team at the Maternal–Infant Care Research Centre for their tireless and diligent work in coordinating study activities (Charlene Wong, BA, BEd; Sukhy Mahl, HBSc, MBA) and maintaining and managing the study database (Sonny Yeh, BSc). All data analyses were performed by Xiang Y. Ye, MSc, in his role as a senior staff member of the statistical team at the Maternal–Infant Care Research Centre. Analyses were conducted under the guidance of Dr. Shah and Dr. Lee. The authors also thank Ruth Warre, PhD, of the Maternal–Infant Care Research Centre for providing editorial support. With the exception of Dr. Shah and Dr. Lee, all staff members acknowledged here were employees of the Maternal–Infant Care Research Centre and received salaried compensation for their work on this project.
Footnotes
↵* The lists of EPIQ investigators (collaborators), members of the Data Safety and Monitoring Committee, members of the Literature Review Committee and Outcome Group chairs appear in Appendix 1 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.140399/-/DC1).
Competing interests: During the study period, Shoo Lee became Scientific Director of the Institute of Human Development, Child and Youth Health, Canadian Institutes of Health Research (CIHR), and received CIHR grants for other projects. Prakesh Shah is a recipient of an Applied Research Chair in Reproductive and Child Health Services and Policy Research from the CIHR. No other competing interests were declared.
This article has been peer reviewed.
Contributors: Shoo Lee conceived the study, obtained the funding, led the project and participated in statistical analysis and interpretation. Prakesh Shah, Nalini Singhal, Khalid Aziz, Anne Synnes, Douglas McMillan and Mary Seshia were involved in study conception and design, study implementation and acquisition of data, and data interpretation; Prakesh Shah was also involved in the statistical analysis. Prakesh Shah drafted the manuscript, and all authors reviewed and revised the manuscript for important intellectual content. All authors approved the final manuscript for submission. Shoo Lee and Prakesh Shah had full access to the study data, take responsibility for data integrity and accuracy of the data analysis, and had final responsibility for the decision to submit for publication. All authors agree to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.
Funding: This work was supported by the Canadian Institutes of Health Research through a grant to the CIHR Team in Maternal–Infant Care (CTP 87518). The study coordinating centre, the Maternal–Infant Care Research Centre, is supported by program funding from the Ontario Ministry of Health and Long-Term Care. In addition, participating sites contributed additional funding for implementation of individual practice changes as required. The funding agencies had no role in the design and conduct of the study; the collection, management, analysis or interpretation of the data; the preparation, review or approval of the manuscript; or the decision to submit the manuscript for publication. The study protocol is available as Appendix 3 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.140399/-/DC1).
References
In this issue
Article tools
Jump to section
Related Articles
Cited By...
- Continued improvement in morbidity reduction in extremely premature infants
- Development of a national neonatal intensive care unit-specific antimicrobial stewardship programme in Canada: protocol for a cohort study
- Trends in sex-specific differences in outcomes in extreme preterms: progress or natural barriers?
- Outcomes and care practices for preterm infants born at less than 33 weeks gestation: a quality-improvement study
- Cognitive trajectories from infancy to early adulthood in the EPICure cohort: time to refocus research on how to break the 'natural limits of brain plasticity
- Incidence of neonatal necrotising enterocolitis in high-income countries: a systematic review
- The Canadian Preterm Birth Network: a study protocol for improving outcomes for preterm infants and their families
- The History and State of Neonatal Nursing Quality Improvement Practice and Education
- Determinants of developmental outcomes in a very preterm Canadian cohort
- Implementing evidence-based practices improves neonatal outcomes
- A Multifaceted Approach to Improving Outcomes in the NICU: The Pediatrix 100 000 Babies Campaign
More in this TOC Section
Similar Articles
Collections
