Validation and refinement of a clinical decision rule for the use of computed tomography in children with minor head injury in the emergency department
=======================================================================================================================================================

* Martin H. Osmond
* Terry P. Klassen
* George A. Wells
* Jennifer Davidson
* Rhonda Correll
* Kathy Boutis
* Gary Joubert
* Serge Gouin
* Simi Khangura
* Troy Turner
* Francois Belanger
* Norm Silver
* Brett Taylor
* Janet Curran
* Ian G. Stiell

## Abstract

**BACKGROUND:** There is uncertainty about which children with minor head injury need to undergo computed tomography (CT). We sought to prospectively validate the accuracy and potential for refinement of a previously derived decision rule, Canadian Assessment of Tomography for Childhood Head injury (CATCH), to guide CT use in children with minor head injury.

**METHODS:** This multicentre cohort study in 9 Canadian pediatric emergency departments prospectively enrolled children with blunt head trauma presenting with a Glasgow Coma Scale score of 13–15 and loss of consciousness, amnesia, disorientation, persistent vomiting or irritability. Phys icians completed standardized assessment forms before CT, including clinical predictors of the rule. The primary outcome was neurosurgical intervention and the secondary outcome was brain injury on CT. We calculated test characteristics of the rule and used recursive partitioning to further refine the rule.

**RESULTS:** Of 4060 enrolled patients, 23 (0.6%) underwent neurosurgical intervention, and 197 (4.9%) had brain injury on CT. The original 7-item rule (CATCH) had sensitivities of 91.3% (95% confidence interval [CI] 72.0%–98.9%) for neurosurgical intervention and 97.5% (95% CI 94.2%–99.2%) for predicting brain injury. Adding “≥ 4 episodes of vomiting” resulted in a refined 8-item rule (CATCH2) with 100% (95% CI 85.2%–100%) sensitivity for neurosurgical intervention and 99.5% (95% CI 97.2%–100%) sensitivity for brain injury.

**INTERPRETATION:** Among children presenting to the emergency department with minor head injury, the CATCH2 rule was highly sensitive for identifying those children requiring neurosurgical intervention and those with any brain injury on CT. The CATCH2 rule should be further validated in an implementation study designed to assess its clinical impact.

Pediatric head injury accounts for more than 500 000 emergency department visits annually in the United States.1 Most head injuries are mild and do not require neurosurgical management. However, those with a history of loss of consciousness, amnesia or disorientation (so-called minor head injury) are a higher-risk group, with 0.5% requiring neurosurgical intervention and 4%–7% having a visible injury on computed tomography (CT).2–6

The use of CT for minor head injury has increased rapidly and is highly variable.5–9 In Canadian pediatric emergency departments, the rate of CT use has increased from 15% in 1995 to 53% in 2005.5,6 This increase exposes many children to the potentially harmful effects of ionizing radiation.10–13 Computed tomography use varies widely across North American hospitals, indicating a lack of standardization in ordering.5,7–9 Two studies reported a recent decline in CT use in children,14,15 but general emergency departments, where most children receive treatment, consistently have CT rates higher than pediatric emergency departments.7,16 Despite this, a small number of intracranial hematomas are missed at the first visit.17

We derived a clinical decision rule in a study involving 3866 children with minor head injury and a Glasgow Coma Scale (GCS)18 score of 13–15.6 The resulting Canadian Assessment of Tomography for Childhood Head injury (CATCH) rule comprises 7 simple variables predicting the need for neurosurgical intervention and brain injury on CT (Box 1). Decision rule validation is an important step before clinical use.19–22 The primary aim of this study was to prospectively validate the accuracy, reliability and acceptability of the CATCH rule in a new cohort of children with minor head injury. The secondary aim was to explore the potential to refine the rule to improve performance.

Box 1: 
### **Canadian Assessment of Tomography for Childhood Head injury (CATCH) rule**

**CT of the head is required for children with minor head injury* and any 1 of these findings:**

High risk for neurosurgical intervention

*   GCS score < 15 at 2 hours after injury

*   Suspected open or depressed skull fracture

*   History of worsening headache

*   Irritability on examination

Medium risk for brain injury on CT

*   Any sign of basal skull fracture†

*   Large, boggy hematoma of the scalp

*   Dangerous mechanism of injury‡

Note: CT = computed tomography, GCS = Glasgow Coma Scale.

*   * Minor head injury defined as injury within the past 24 hours associated with witnessed loss of consciousness, definite amnesia, witnessed disorientation, persistent vomiting (> 1 episode) or persistent irritability (in a child aged < 2 yr) in a patient with a GCS score of 13–15.

*   † Signs of basal skull fracture include hemotympanum, raccoon eyes, otorrhea or rhinorrhea of the cerebrospinal fluid, and Battle sign.

*   ‡ Dangerous mechanism is a motor vehicle crash, a fall from an elevation ≥ 3 ft (≥ 91 cm) or 5 stairs, or a fall from a bicycle with no helmet.

## Methods

### Study design and setting

This was a prospective multicentre cohort study in the emergency departments of 9 Pediatric Emergency Research Canada (PERC) member hospitals from April 2006 through December 2009.

### Study population

Children (aged 0–16 yr) with acute head injury were eligible for enrolment if they had all of the following: blunt head trauma resulting in witnessed loss of consciousness, amnesia, disorientation, persistent vomiting (≥ 2 episodes of vomiting 15 min apart) or persistent irritability for children 2 years of age or younger; initial emergency department GCS score of 13 or greater; and injury within the past 24 hours. Patients were ineligible if they had obvious penetrating skull injury or depressed fracture; they had acute focal neurologic deficit; they had chronic generalized developmental delay; child abuse was suspected; they returned for reassessment of the same head injury; or they were pregnant. Written informed consent was obtained at the time of enrolment.

### Standardized patient assessment

Emergency staff physicians or senior residents made all patient assessments. Physicians underwent a 1-hour lecture describing the study and the standardized assessments. After assessing patients but before ordering any imaging, physicians recorded 15 clinical findings on data forms. This included variables in the CATCH rule and other clinically important variables for possible rule refinement. Physicians also documented their interpretation of the CATCH rule and their degree of comfort in using the rule. When feasible, a second physician completed an independent assessment to allow determination of interobserver agreement. Physicians were instructed to use their clinical judgment and not the CATCH rule when deciding when to order CT.

Computed tomography scans were interpreted by staff radiologists, who were blinded to patient assessment forms, at each site as part of routine clinical operations. If the radiologist expressed uncertainty about whether an acute intracranial injury existed, the CT scan was reviewed by another radiologist and a neurosurgeon who were both blinded. If uncertainty still existed, the scan was considered negative.

Because not all children with minor head injury routinely undergo CT, we could not ethically mandate universal CT for all patients. Patients who did not undergo imaging were classified as having no clinically important brain injury if they met all the following criteria at 14 days when contacted by telephone: headache absent or mild; no memory or concentration problems; no seizure or focal motor findings; and return to normal daily activities. These criteria were assessed by a nurse who was blinded to the clinical details of the patient. Patients not fulfilling these criteria were recalled for clinical reassessment and CT. Patients could only be classified as having brain injury based on their CT findings. Patients who did not undergo CT and could not be reached were excluded from the final analysis.

### Outcome measures

The primary outcome was neurosurgical intervention, defined as either death within 7 days secondary to head injury or the need for any of the following procedures within 7 days: craniotomy, elevation of skull fracture, intracranial pressure monitoring or intubation for head injury (Box 2).

Box 2: 
### **Definitions of traumatic brain injury outcomes**

**Primary outcome: neurosurgical intervention***

Defined by any of the following descriptions:

*   Death within 7 days due to traumatic brain injury

*   Need for any of the following procedures within 7 days:
    
    *   craniotomy
    
    *   elevation of depressed skull fracture
    
    *   intracranial pressure monitoring
    
    *   intubation for the treatment of head injury

**Secondary outcome: brain injury on CT†**

Defined by any of the following descriptions:

*   Intracranial hemorrhage or contusion, including
    
    *   epidural hematoma
    
    *   subdural hematoma
    
    *   subarachnoid hemorrhage
    
    *   intracerebral hematoma
    
    *   cerebral contusion
    
    *   cerebellar hematoma
    
    *   intraventricular hemorrhage
    
    *   extra-axial hematoma — undifferentiated

*   Diffuse cerebral edema

*   Pneumocephalus

*   Depressed skull fracture (depressed past the inner table of the skull)

Note: CT = computed tomography.

*   * The neurosurgical intervention group requires prompt recognition and an intervention to treat the traumatic brain injury.

*   † The brain injury on CT group usually requires admission to hospital for observation and neurologic follow-up.

The secondary outcome was brain injury on CT, defined as any acute intracranial finding on CT attributable to acute trauma. This included closed depressed skull fractures (depressed past the inner table) and pneumocephalus, but excluded nondepressed skull fractures and basilar skull fractures.

### Statistical analysis

We examined demographic and clinical characteristics using simple descriptive statistics. We assessed interobserver agreement with the κ statistic. Univariate analyses used the 2-sided *t* test for continuous variables and the Pearson χ2 test for categorical variables. We evaluated the accuracy of the interpretation of the rule by the treating physician versus the criterion interpretation by the investigators. The criterion interpretation of the rule (i.e., whether positive or negative for the outcome measures) was made by an adjudication committee that reviewed patient records and physician responses on data forms.

Potential refinement of the rule was assessed by multivariate recursive partitioning analysis using Knowledge SEEKER version 8.7. The sensitivity, specificity and 95% confidence intervals (CIs) were calculated for the refined rule. We estimated that a sample size of 4000 patients with minor head injury would be required to provide a sufficient number of brain injury cases to allow a 95% CI of 96%–100% around a sensitivity of 99%. We conducted a post hoc bootstrapping analysis23 to determine the internal stability of the refined CATCH rule using our previous derivation cohort (*n* = 3866). The first 1000 samples of the full size of the data set were selected using sampling with replacement. Estimates of the sensitivity and specificity were calculated for each sample, and the mean of these estimates was used as the bootstrap sensitivity and specificity estimates. We used the 2.5 and 97.5 percentile of the 1000 estimates as the lower and upper bound for the 95% CI of the bootstrap estimates. Analyses were conducted using SAS version 9.3.

### Ethics approval

The research ethics committee of each hospital approved the study. The research ethics board of the Children’s Hospital of Eastern Ontario gave the first approval.

## Results

Between April 2006 and December 2009, a total of 6525 eligible patients were seen in the 9 study hospitals. Of these, 4494 (68.9%) were enrolled, 1609 (24.7%) were missed eligible patients and 422 (6.5%) did not consent to the study (Figure 1). Of the 4494 enrolled, 4060 (90.3%) had data forms completed by physicians and complete outcome assessments documented. The remaining 434 (9.7%) patients had no CT scan and no proxy outcome measure as they could not be reached for follow-up by telephone despite multiple attempts.

![Figure 1:](http://www.cmaj.ca/https://www.cmaj.ca/content/cmaj/190/27/E816/F1.medium.gif)

[Figure 1:](http://www.cmaj.ca/content/190/27/E816/F1)

Figure 1: 
Study flow. CT = computed tomography.

Table 1 shows the clinical characteristics of the 4060 enrolled patients. Patients ranged in age from 1 month to 16 years with a mean age of 9.7 years, and 464 (11.4%) were younger than 2 years. The characteristics of the 1609 missed potentially eligible patients were similar to those of patients who were enrolled, including in mean age (9.5 v. 9.7 yr), male sex (65.6% v. 64.5%), rate of arrival by ambulance (38.4% v. 36.1%), transfer from another hospital (13.1% v. 12.5%) and mechanism of injury.

View this table:
[Table 1:](http://www.cmaj.ca/content/190/27/E816/T1)

Table 1: 
Characteristics of 4060 children with head injury

Table 2 reports the management and outcomes of the enrolled patients. A total of 1417 (34.9%) patients underwent CT, and the remaining 2643 (65.1%) patients, who were all discharged from the emergency department, underwent the 14-day telephone follow-up. Of those, 20 patients were recalled by study nurses, and 19 of these patients underwent CT, which were all negative. Of all cases, 23 (0.6%) patients underwent neurosurgical intervention (with 20 receiving a craniotomy), and 197 (4.9%) patients were found to have a brain injury on CT.

View this table:
[Table 2:](http://www.cmaj.ca/content/190/27/E816/T2)

Table 2: 
Management and outcomes of 4060 patients with head injury

Table 3 shows the performance of the predictor variables in patients with and without brain injury, as determined by univariate analyses and interobserver agreement. All 7 variables of the CATCH rule were highly associated with acute brain injury (*p* < 0.001). Other variables strongly associated with brain injury included repeated vomiting (≥ 3 and ≥ 4 episodes), initial GCS score of 13, scalp hematoma and lethargy on examination. The κ value for the overall interpretation of the CATCH rule was 0.67 (95% CI 0.60–0.74).

View this table:
[Table 3:](http://www.cmaj.ca/content/190/27/E816/T3)

Table 3: 
Interobserver agreement and univariate correlation of variables for brain injury (*n* = 4060)

The classification performance of the CATCH rule is shown in Table 4. For detecting patients requiring neurosurgical intervention, the sensitivity was 91.3% and the specificity was 57.1%. For detecting patients with brain injury on CT, the sensitivity was 97.5% and the specificity was 59.6%.

View this table:
[Table 4:](http://www.cmaj.ca/content/190/27/E816/T4)

Table 4: 
Sensitivity and specificity of the original 7-item CATCH rule for children with minor head injury

Rule refinement by recursive partitioning showed that the best model to predict all cases of neurosurgical intervention and all but 1 case of brain injury on CT retained all the variables in the original 7-item CATCH rule and added the variable “vomiting ≥ 4 episodes.” This variable describes the number of episodes of vomiting (> 15 min apart) that occurred since the head trauma. The classification performance of this new rule is shown in Table 5. Sensitivity for detecting patients requiring neurosurgical intervention using the new rule was 100% (95% CI 85.2%–100%) and the specificity was 45.7% (95% CI 44.2%–47.3%). Sensitivity for detecting brain injury on CT was 99.5% (95% CI 97.2%–100%) and the specificity was 47.8% (95% CI 46.8%–49.4%). We designated this refined 8-item rule the CATCH2 rule, shown in Box 3.

View this table:
[Table 5:](http://www.cmaj.ca/content/190/27/E816/T5)

Table 5: 
Sensitivity and specificity of the 8-item CATCH2 rule for children with minor head injury

Box 3: 
### **The Canadian Assessment of Tomography for Childhood Head injury 2 (CATCH2) rule**

**CT of the head is required for children with minor head injury* and any 1 of these findings:**

*   GCS score < 15 at 2 hours after injury

*   Suspected open or depressed skull fracture

*   History of worsening headache

*   Irritability on examination

*   Any sign of basal skull fracture†

*   Large, boggy hematoma of the scalp

*   Dangerous mechanism of injury‡

*   ≥ 4 episodes of vomiting

Note: CT = computed tomography, GCS = Glasgow Coma Scale.

*   * Minor head injury is defined as injury within the past 24 hours associated with witnessed loss of consciousness, definite amnesia, witnessed disorientation, persistent vomiting (> 1 episode) or persistent irritability (in a child aged < 2 yr) in a patient with a GCS score of 13–15.

*   † Signs of basal skull fracture include hemotympanum, raccoon eyes, otorrhea or rhinorrhea of the cerebrospinal fluid, and Battle sign.

*   ‡ Dangerous mechanism is a motor vehicle crash, a fall from elevation ≥ 3 ft (≥ 91 cm) or 5 stairs, or a fall from a bicycle with no helmet.

Our bootstrap analysis (1000 replications) for the CATCH2 rule using the derivation data6 had a sensitivity of 100% (95% CI 100%–100%) and a specificity of 35.6% (95% CI 34.0%–37.1%) for neurosurgical intervention, whereas the sensitivity for brain injury on CT was 99.4% (95% CI 97.9%–100%) and the specificity was 36.9% (95% CI 35.2%–38.4%).

The potential impact of the rule was assessed by comparing the CT rate according to the CATCH rule to the actual clinical practice of the physicians at the 9 sites. For the 4060 enrolled patients, the CT rate according to the CATCH rule was 43% and the CT ordering rate by the physicians was 35%. Using the CATCH2 rule, the CT ordering rate would be 55%.

One case of brain injury would have been missed using the CATCH2 rule. This was an adolescent whose occiput struck the ground after a tackle. The patient had loss of consciousness for 2 minutes, amnesia to the event and a moderate headache (not worsening). Computed tomography on presentation to the emergency department showed a 1 mm thick acute subdural bleed. The patient was discharged directly home from the emergency department after observation.

Physicians reported they would be comfortable to very comfortable using the rule in 3295 cases (81.5%), neutral in 347 cases (8.6%), and uncomfortable to very uncomfortable in 399 cases (9.9%). In an evaluation of accuracy, treating physicians misinterpreted the rule as not requiring imaging when the clinical criteria suggested imaging in 8.7% of patients. The most frequently misinterpreted variables were dangerous mechanism of injury and irritability.

## Interpretation

This validation study shows that the 7-item CATCH rule had inadequate sensitivity to detect those children requiring neurosurgical intervention. The rule was refined by adding a variable for recurrent vomiting to create the 8-item CATCH2 rule, which achieved 100% sensitivity for neurosurgical intervention and 99.5% sensitivity for detecting brain injury on CT. The trade-off for this improved sensitivity is decreased specificity. Following this rule would result in 55% of patients with minor head injury undergoing CT.

The variables that make up the CATCH2 rule have been found to be significant predictors of intracranial injury in previous observational studies of pediatric head injury24–29 including a GCS score of less than 15 at 2 hours,25–29 suspected open or depressed skull fracture,25–29 worsening headache, 24,25 persistent irritability,26 sign of basal skull fracture,25–29 large boggy scalp hematoma,26,27 recurrent vomiting26,27,29 and dangerous mechanism.27–29 This consistency between studies supports the importance of these risk factors in predicting intracranial injury in children.

The Pediatric Emergency Care Applied Research Network (PECARN) developed decision rules to identify children at low risk of clinically important traumatic brain injury who could be safely sent home without CT.29 Although use of these decision rules may reduce the use of CT in children with minimal symptoms, these rules do not indicate which children with more concerning signs and symptoms require CT. Interestingly, the PECARN study does identify “altered mental status” and “signs of skull fracture” as the highest risk clinical factors of clinically important traumatic brain injury, which is consistent with 5 variables of the CATCH2 rule.

Babl and colleagues30 recently published a well-designed prospective external validation of the CATCH,6 PECARN29 and CHALICE (Children’s Head Injury Algorithm for the Prediction of Important Clinical Events)27 rules. They reported a high sensitivity (95.2%, 95% CI 76.2%–99.9%) for the CATCH rule, which predicted 20 of 21 patients requiring neurosurgical intervention. The 1 missed patient had repeated vomiting (> 3 times) and would have been identified by the new CATCH2 rule had it been available for their validation study.

We found that the rate of CT use for children with minor head injury has decreased significantly between the time of our derivation study (2001–2005), when the rate was 53%, and our validation study (2006–2009), when the rate was 35%.6 We believe that raised awareness of the risks associated with radiation10,11,13 involved with CT has already resulted in pediatric emergency physicians becoming more selective with whom they choose to undergo CT.

### Limitations

This study has potential limitations. Computed tomography was not performed on all enrolled patients because we could not ethically justify exposing children to radiation when their treating clinician felt CT was not required. However, the patients who did not receive CT underwent a validated telephone interview at 14 days for determination of the proxy outcome measure. Although 9.7% of patients were lost to follow-up, this rate is similar to those of other large prospective emergency department studies of head injury in children, such as the studies by Kuppermann and colleagues29 and Babl and colleagues,30 which had lost-to-follow-up rates of 21% and 10%, respectively. Although the interobserver agreement (κ = 0.67) for the CATCH rule is considered moderate to substantial,31–33 future implementation studies should focus on improving this.

Bootstrapping procedures for exploring uncertainty in model development of the CATCH2 rule may be informative but could not be followed because the specific classification and regression tree approach used for developing this rule does not lend itself to repeated model building. We did apply the CATCH2 rule to the derivation data set using a bootstrapping technique to provide a more robust evaluation of the sensitivity and specificity of the rule and to show that the slight revision is not of such magnitude to invalidate the rule. As expected, the revised rule was validated. However, the formal verification of the validation of the CATCH2 rule will come through applying it to a new prospective data set during the subsequent implementation phase.

The decision to obtain a CT scan is affected by many factors, including setting, number and progression of symptoms, patient and parent preference, and physician experience. Although the sensitivity of the CATCH2 rule is very high, it is not perfect. The rule was developed according to strict methodologic standards19,20 to assist physicians in making decisions regarding immediate CT, prompt referral to a centre with CT availability or need for continued close observation in the emergency department. The effect on CT rates will depend on the emergency department setting and local practice patterns; however, using this rule should result in exceedingly few missed injuries.

### Conclusion

Among children presenting to the emergency department with the signs and symptoms of acute minor head injury, the CATCH2 rule was highly sensitive for identifying those children requiring neurosurgical intervention and those with any brain injury on CT. The CATCH2 rule should be further validated in an implementation study designed to assess its clinical impact.

## Acknowledgements

The authors thank the site research coordinators of Pediatric Emergency Research Canada, whose hard work and dedication made this study possible. The authors thank My-Linh Tran and Sheryl Domingo for data management, and Jordan Bernick for statistical advice. The authors also thank all the emergency physicians, nurses and clerks at the study sites who voluntarily assisted with case identification and data collection, and all the site radiologists who interpreted the computed tomography images.

## Footnotes

*   **Competing interests:** None declared.

*   This article has been peer reviewed.

*   **Contributors:** Martin Osmond, Terry Klassen, Ian Stiell and George Wells conceived the study. All of the authors contributed to the conceptual design of the study and the application for research funding. Rhonda Correll, Jennifer Davidson, Janet Curran, Kathy Boutis, Gary Joubert, Serge Gouin, Simi Khangura, Troy Turner, Francois Belanger, Norm Silver and Brett Taylor contributed to the recruitment of patients and the acquisition of data. Martin Osmond, Terry Klassen, George Wells, Janet Curran and Ian Stiell contributed to the analysis and interpretation of the data and to drafting the manuscript. All of the authors critically reviewed the manuscript, contributed substantially to its revision, approved the final version submitted for publication and agreed to act as guarantors of the work.

*   **Funding:** This study was funded by a peer-reviewed grant from the Canadian Institutes of Health Research (CIHR funding reference no. MOP-43911).

*   **Disclaimer:** George Wells is a biostatistical consultant for *CMAJ* and was not involved in the editorial decision-making process for this article.

*   Accepted March 13, 2018.

## References

1.  Faul M, Xu L, Wald MM, et al. Traumatic brain injury in the United States: emergency department visits, hospitalizations and deaths 2002–2006. Atlanta: U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Injury Prevention and Control; 2010.
    
    

2.  Mitchell KA, Fallat ME, Raque GH, et al. Evaluation of minor head injury in children. J Pediatr Surg 1994;29:851–4.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/0022-3468(94)90001-9&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=7931956&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1994NX18000001&link_type=ISI) 

3.  Quayle KS, Jaffe DM, Kuppermann N, et al. Diagnostic testing for acute head injury in children: When are head computed tomography and skull radiographs indicated? Pediatrics 1997;99:E11.
    
    

4.  Davis RL, Hughes M, Gubler KD, et al. The use of cranial CT scans in the triage of pediatric patients with mild head injury. Pediatrics 1995;95:345–9.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6ODoiOTUvMy8zNDUiO3M6NDoiYXRvbSI7czoyMjoiL2NtYWovMTkwLzI3L0U4MTYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

5.  Klassen TP, Reed MH, Stiell IG, et al. Variation in utilization of computed tomography scanning for the investigation of minor head trauma in children: a Canadian experience. Acad Emerg Med 2000;7:739–44.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1111/j.1553-2712.2000.tb02260.x&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10917321&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000088012200002&link_type=ISI) 

6.  Osmond MH, Klassen TP, Wells GA, et al.Pediatric Emergency Research Canada (PERC) Head Injury Study Group. CATCH: a clinical decision rule for the use of computed tomography in children with minor head injury. CMAJ 2010;182: 341–8.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6NDoiY21haiI7czo1OiJyZXNpZCI7czo5OiIxODIvNC8zNDEiO3M6NDoiYXRvbSI7czoyMjoiL2NtYWovMTkwLzI3L0U4MTYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

7.  Blackwell CD, Gorelick M, Holmes JF, et al. Pediatric head trauma: changes in use of computed tomography in emergency departments in the United States over time. Ann Emerg Med 2007;49:320–4.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.annemergmed.2006.09.025&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=17145113&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000244637000013&link_type=ISI) 

8.  Mannix R, Meehan WP, Monuteaux MC, et al. Computed tomography for minor head injury: variation and trends in major United States pediatric emergency departments. J Pediatr 2012;160:136–9.e1.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.jpeds.2011.06.024&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=21813133&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000298143000031&link_type=ISI) 

9.  Stanley RM, Hoyle JD Jr., Dayan PS, et al.Pediatric Emergency Care Applied Research Network (PECARN). Emergency department practice variation in computed tomography use for children with minor blunt head trauma. J Pediatr 2014;165:1201–6.e2.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.jpeds.2014.08.008&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=25294604&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 

10. Brenner DJ, Elliston CD, Hall EJ, et al. Estimated risks of radiation-induced fatal cancer from pediatric CT. AJR Am J Roentgenol 2001;176:289–96.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.2214/ajr.176.2.1760289&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=11159059&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000166594100002&link_type=ISI) 

11. Brenner DJ, Hall EJ. Computed tomography — an increasing source of radiation exposure. N Engl J Med 2007;357:2277–84.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJMra072149&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=18046031&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000251176400009&link_type=ISI) 

12. Hall P, Adami H-O, Trichopoulos D, et al. Effect of low doses of ionising radiation in infancy on cognitive function in adulthood: Swedish population based cohort study. BMJ 2004;328:19.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MzoiYm1qIjtzOjU6InJlc2lkIjtzOjExOiIzMjgvNzQzMC8xOSI7czo0OiJhdG9tIjtzOjIyOiIvY21hai8xOTAvMjcvRTgxNi5hdG9tIjt9czo4OiJmcmFnbWVudCI7czowOiIiO30=) 

13. Pearce MS, Salotti JA, Little MP, et al. Radiation exposure from CT scans in childhood and subsequent risk of leukaemia and brain tumours: a retrospective cohort study. Lancet 2012;380:499–505.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(12)60815-0&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=22681860&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000307109000031&link_type=ISI) 

14. Menoch MJA, Hirsh DA, Khan NS, et al. Trends in computed tomography utilization in the pediatric emergency department. Pediatrics 2012;129:e690–7.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6MTA6IjEyOS8zL2U2OTAiO3M6NDoiYXRvbSI7czoyMjoiL2NtYWovMTkwLzI3L0U4MTYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

15. Berdahl CT, Vermeulen MJ, Larson DB, et al. Emergency department computed tomography utilization in the United States and Canada. Ann Emerg Med 2013; 62:486–94.e3.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/j.annemergmed.2013.02.018&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23683773&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 

16. Mannix R, Bourgeois FT, Schutzman SA, et al. Neuroimaging for pediatric head trauma: Do patient and hospital characteristics influence who gets imaged? Acad Emerg Med 2010;17:694–700.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1111/j.1553-2712.2010.00797.x&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=20653582&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000279613400010&link_type=ISI) 

17. Hamilton M, Mrazik M, Johnson DW. Incidence of delayed intracranial hemorrhage in children after uncomplicated minor head injuries. Pediatrics 2010; 126:e33–9.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6OToiMTI2LzEvZTMzIjtzOjQ6ImF0b20iO3M6MjI6Ii9jbWFqLzE5MC8yNy9FODE2LmF0b20iO31zOjg6ImZyYWdtZW50IjtzOjA6IiI7fQ==) 

18. Teasdale G, Jennett B. Assessment of coma and impaired consciousness: a practical scale. Lancet 1974;2:81–4.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(02)93219-8&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=4136544&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1974T535500009&link_type=ISI) 

19. Laupacis A, Sekar N, Stiell IG. Clinical prediction rules: a review and suggested modifications of methodological standards. JAMA 1997;277:488–94.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/jama.1997.03540300056034&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=9020274&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1997WG05400037&link_type=ISI) 

20. Stiell IG, Wells GA. Methodologic standards for the development of clinical decision rules in emergency medicine. Ann Emerg Med 1999;33:437–47.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0196-0644(99)70309-4&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=10092723&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000079433500011&link_type=ISI) 

21. Wasson JH, Sox HC, Neff RK, et al. Clinical prediction rules: application and methodological standards. N Engl J Med 1985;313:793–9.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1056/NEJM198509263131306&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=3897864&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1985ARA8900006&link_type=ISI) 

22. Charlson ME, Ales KL, Simon R, et al. Why predictive indexes perform less well in validation studies. Arch Intern Med 1987;147:2155–61.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/archinte.1987.00370120091016&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=3689067&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1987L133500011&link_type=ISI) 

23. Efron B, Tibshirani R. An introduction to the bootstrap. 1st ed. London (UK): Chapman and Hall; 1993.
    
    

24. Haydel MJ, Shembekar AD. Prediction of intracranial injury in children aged five years and older with loss of consciousness after minor head injury due to nontrivial mechanisms. Ann Emerg Med 2003;42:507–14.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1067/S0196-0644(03)00512-2&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=14520321&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000185752700007&link_type=ISI) 

25. Palchak MJ, Holmes JF, Vance CW, et al. A decision rule for identifying children at low risk for brain injuries after blunt head trauma. Ann Emerg Med 2003;42: 492–506.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1067/S0196-0644(03)00425-6&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=14520320&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000185752700006&link_type=ISI) 

26. Oman JA, Cooper RJ, Holmes JF, et al.NEXUS II Investigators. Performance of a decision rule to predict need for computed tomography among children with blunt head trauma. Pediatrics 2006;117:e238–46.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTA6InBlZGlhdHJpY3MiO3M6NToicmVzaWQiO3M6MTA6IjExNy8yL2UyMzgiO3M6NDoiYXRvbSI7czoyMjoiL2NtYWovMTkwLzI3L0U4MTYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

27. Dunning J, Daly JP, Lomas JP, et al. Children’s head injury algorithm for the prediction of important clinical events study group. Derivation of the children’s head injury algorithm for the prediction of important clinical events decision rule for head injury in children. Arch Dis Child 2006;91:885–91.
    
    [Abstract/FREE Full Text](http://www.cmaj.ca/lookup/ijlink/YTozOntzOjQ6InBhdGgiO3M6MTQ6Ii9sb29rdXAvaWpsaW5rIjtzOjU6InF1ZXJ5IjthOjQ6e3M6ODoibGlua1R5cGUiO3M6NDoiQUJTVCI7czoxMToiam91cm5hbENvZGUiO3M6MTI6ImFyY2hkaXNjaGlsZCI7czo1OiJyZXNpZCI7czo5OiI5MS8xMS84ODUiO3M6NDoiYXRvbSI7czoyMjoiL2NtYWovMTkwLzI3L0U4MTYuYXRvbSI7fXM6ODoiZnJhZ21lbnQiO3M6MDoiIjt9) 

28. Atabaki SM, Stiell IG, Bazarian JJ, et al. A clinical decision rule for cranial computed tomography in minor pediatric head trauma. Arch Pediatr Adolesc Med 2008;162:439–45.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1001/archpedi.162.5.439&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=18458190&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000255559900008&link_type=ISI) 

29. Kuppermann N, Holmes JF, Dayan PS, et al.Pediatric Emergency Care Applied Research Network (PECARN). Identification of children at very low risk of clinically-important brain injuries after head trauma: a prospective cohort study. Lancet 2009;374:1160–70.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(09)61558-0&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=19758692&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=000270688200032&link_type=ISI) 

30. Babl FE, Borland ML, Phillips N, et al.Paediatric Research in Emergency Departments International Collaborative (PREDICT). Accuracy of PECARN, CATCH, and CHALICE head injury decision rules in children: a prospective cohort study. Lancet 2017;389:2393–402.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.1016/S0140-6736(17)30555-X&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=28410792&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 

31. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159–74.
    
    [CrossRef](http://www.cmaj.ca/lookup/external-ref?access_num=10.2307/2529310&link_type=DOI) 
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=843571&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom) 
    
    [Web of Science](http://www.cmaj.ca/lookup/external-ref?access_num=A1977CY39700012&link_type=ISI) 

32. Fleiss JL. Statistical methods for rates and proportions. 2nd ed. New York: John Wiley; 2000: 218.
    
    

33. McHugh ML. Interrater reliability: the kappa statistic. Biochem Med (Zagreb) 2012;22:276–82.
    
    [PubMed](http://www.cmaj.ca/lookup/external-ref?access_num=23092060&link_type=MED&atom=%2Fcmaj%2F190%2F27%2FE816.atom)