Article Text
PDF
Abstract
Traumatic brain injury (TBI) is the commonest cause of acquired disability in childhood. A major obstacle to the evaluation of acute and rehabilitative therapies after TBI is the lack of simple descriptors of outcome. We developed the King's Outcome Scale for Childhood Head Injury (KOSCHI), as a specific paediatric adaptation of the original adult Glasgow Outcome Scale (GOS). The KOSCHI expands the five category GOS to provide increased sensitivity at the milder end of the disability range. The GOS category of “persistent vegetative state” was replaced by “vegetative”. “Good recovery” was allocated two categories, in acknowledgement of the long term importance of relatively minor sequelae in a developing child. The scale was quick and easy to use. Inter-rater reliability studies show that even with such an apparently simple scale, some training may be required. The KOSCHI provides a practical scale for paediatric head injury which will enable clinicians to describe rate and extent of recovery, and evaluate the effects of service and research interventions.
- head injury
- traumatic brain injury
- outcome scale
Statistics from Altmetric.com
When Jennett and Bond published the Glasgow Outcome Scale (GOS) in 1975,1 they made particular mention of the special developmental considerations in the assessment of outcome of traumatic brain injury (TBI) in children. Although the GOS has since become a standard outcome descriptor in adult TBI there remains no equivalent scale for use in children.
Head injury accounts for 5% of all paediatric hospital admissions and is the commonest cause of acquired disability in childhood, with an estimated 3000 children acquiring significant new neurological or cognitive disability as a result of TBI every year in the UK.2 The sequelae are wide ranging and often include problems with behaviour and learning, even in the absence of physical deficits.3
A particular problem in the field of paediatric TBI research is the difficulty in developing simple yet meaningful descriptors of outcome. There are several challenges, particularly in describing outcome at the “higher” levels of functional independence or participation in society (see Discussion). We set out to produce a modification of the GOS which would provide a robust, simple description of outcome after paediatric TBI in the short, medium, or long term.
Methods
Information about the nature of head injury sequelae at various stages of recovery was obtained during the routine inpatient management and outpatient follow up of approximately 200 head injured children admitted to the paediatric wards of King's College Hospital and the paediatric neurosurgical unit over the period 1990–1997. More than 90% had severe injury with abnormal intracerebral radiological findings on computed tomography. A simple checklist (table 1) was developed as an “aide memoire” for outpatient follow up. An hour long semistructured interview was held with the parents of six of the more severely injured children (aged 6–14 years) being followed three to five years after TBI to assess the extent and nature of problems encountered in the longer term. On the basis of this experience we modified the five categories of the GOS to produce the King's Outcome Scale for Childhood Head Injury (KOSCHI) (table2).
TBI follow up checklist
KOSCHI category definitions
NOTES ON PARTICULAR CATEGORIES
Death has to be directly attributable to the head injury and includes brain stem death.4 The adjective “persistent”, with its connotations of duration to date and implications of future persistence has been abandoned in connection with the vegetative state (category 2). All children pass through the vegetative stage, albeit often briefly, as they recover consciousness.
The original GOS category of severe disability has been divided into two. A child in category 3a will show some evidence of awareness (for example, visual tracking) although responsiveness (for example, to command) may be unreliable. In category 3b the child has some limited functional independence but requires adult help or supervision for most activities. He may have simple, yet meaningful, possibly non-verbal communication. As with the GOS, severe cognitive damage may place a child in one of these categories despite good motor recovery.
The GOS defines “good recovery” (categories 5a, 5b) in terms of the resumption of previous life (although not necessarily full return to work), notwithstanding minor neurological or psychological deficits. However, the authors acknowledge that head injury in childhood may produce deficits which increase with continuing development. Minor residual deficits are potentially much more destructive to children, and our scale acknowledges the importance of this, particularly in relation to social and learning behaviour. We therefore have stricter criteria for this category and have divided the scale into two subcategories (see table 2).
INTER-RATER RELIABILITY
An extended inter-rater reliability exercise was performed. Six observers with professional interests in childhood head injury (two paediatric neurologists, two clinical nurse specialists, a health economist, and a medical social worker, all with specialist experience in the field) independently assigned KOSCHI categories (from 2 to 5b) to 90 child survivors of head injury. Ratings were made from the information in anonymised routine outpatient follow up clinic letters. Inter-rater reliability was calculated using a modified kappa statistic. The kappa statistic measures the agreement between observers rating samples on a nominal scale. The original Cohen statistic5 considered the case of two observers. Fleiss6 and Dunn7 described analogous statistics for the case of multiple observers; these were used in this study.
Further studies of inter-rater differences required treatment of the KOSCHI category score as a continous scale (see Discussion). See below for justification for this transformation. KOSCHI scores (1 to 5b) were mapped to a continuous scale (1 to 8). Observers' performances were compared by calculating the residual between each observer's rating of each child and the mean of all observers' ratings for that child.
CONSTRUCT VALIDITY
In order to identify the factors determining KOSCHI scale assignments, a simple functional score based on likely candidates was devised (table 3). Using a separate dataset of 80 outpatient clinic letters, children's functional scores (from table 3) and overall KOSCHI category were independently assigned by two observers. Functional scores were related to overall KOSCHI category by multinomial logit and conventional regression.
Components of the functional scoring system
Results
INTER-RATER RELIABILITY
Treating KOSCHI categories as a continuous variable showed a systematic tendency of observer 2 to rate children as more severely disabled; mean of residual scores for observer 2 was −0.52 (SEM 0.08) KOSCHI grades (p < 0.0001; t test). This tendency was replicated in a duplicate analysis of 99 children (data not shown). No other systematic effects were noted (table4).
Observer effects
Treating the KOSCHI categories as a categorical variable, the kappa score for six observers' ratings of 90 children was 0.51. Excluding observer 2, the kappa statistic among the remaining five observers was 0.58. Table 5 shows kappa statistics for each grade. Observer agreement was highest for very severely disabled children. Agreement was poorest in categories 3b and 4a.
Inter-rater reliability
CONSTRUCT VALIDITY
Regression studies of independently assigned functional scores (table 3) and overall KOSCHI category confirmed that it is reasonable to treat KOSCHI categories (1 to 5b) as an eight point interval scale, should this be required. In the dataset of 80 patients used for this analysis, only one was in KOSCHI category 2, and only two in category 3b. These patients were almost completely identified by their mobility, communication, and self care scores. A regression involving the remaining 77 patients showed that the KOSCHI category could be predicted by a simple linear function of the component scores with increased weighting given to the self care score. While the total “functional score” calculated in this way was highly correlated with KOSCHI grade (R = 0.82) the residual standard deviation was considerable at 0.7 of a KOSCHI grade.
Discussion
In the forthcoming revision (ICIDH2) of the WHO Classification of Impairments, Disabilities, and Handicaps, the concepts of activity (functional at the level of the individual, in a standardised, optimal environment) and participation (determined by the interaction between the individual's potential and real world environmental constraints) replace those previously denoted by the terms “disability” and “handicap”. There are several particular challenges in describing outcome after childhood TBI at these higher levels. Measures must relate to age appropriate expectations, and since the developmental process continues (albeit modified) after the injury, the shortfall between current functional status and age dependent “normality” is dynamic. There is also a dynamic relation between the component parts of the total morbidity seen after TBI. Physical, behavioural, and cognitive components make varying contributions to the overall morbidity, depending on age at injury,8 interval since injury, injury severity,9 and injury independent factors.10 ,11
The GOS is completed on the basis of the clinician's assessment of the best available clinical information. We wished to evaluate the KOSCHI in a “real world” setting where scores were being made retrospectively on the basis of information in routine clinic letters. The limited detail with which children's abilities were discussed in these letters was, we feel, responsible in large part for the modest inter-rater reliability. In a research setting, in which KOSCHI gradings are being made on the basis of systematically collected information, inter-rater reliability is likely to be considerably higher.
In general terms, the kappa statistic quantifies the observed degree of interobserver agreement and compares the excess of observed agreement over that expected by chance to the maximum possible excess of observed agreement over chance–expectation. A major disadvantage of the kappa statistic is therefore its dependence on the chance expectation of interobserver agreement, which is itself dependent on the number of observers, observations, and possible categories. Thus conventional criteria of “substantial” (kappa 0.6–0.79) or “almost perfect” agreement (kappa >0.8) relating to two observers are inapplicable in this context. Interpretation of the “adequacy” of the kappa of 0.51–0.58 seen in this study is therefore difficult.
An additional disadvantage of the kappa statistic is that no account is taken of the magnitude of interobserver disagreement. Discrepancy of any magnitude (even one grade step) represents “disagreement”. To overcome this difficulty we performed the continuous variable analyses (table 4). These show the potential for significant systematic errors in rating, even with such an apparently simple instrument as the KOSCHI. Before the KOSCHI is used in clinical or research practice, we would recommend a team of potential observers to perform pilot comparisons of their rating practices to ensure that they are scoring consistently.
Conceptually, the suggestion that “recovery after head injury” can be captured in a unidimensional scale could be challenged. The difference between KOSCHI grades 1 (dead) and 2 (vegetative) is one of physiological function; between 2 and 3a, a matter of awareness of and response to external stimuli; and between grades 3b to 5a essentially degree of functional independence. Nevertheless the multinomial logit regression lends support to use of a unidimensional scale; an approach related to the more commonly used analyses based on Rasch item response theory. The construct validity studies are reassuring. The slightly greater weight given to the self care component is explicable in terms of the criteria defining the grades 3b to 5a, which are largely based on functional independence.
The authors of the GOS recognised the difficulties in applying it to children. We set out to produce a modification of the GOS which would provide a robust, simple scale for short, medium, and long term paediatric head injury outcome. Our requirements included compatibility with the GOS to allow comparison with adults, and facilitate the transition of follow up into adulthood; applicability at all ages; and simplicity of use. It can be completed by direct observation or from routine medical records, prospectively or retrospectively. Sensitivity to apparently minor sequelae of TBI, which in children may have long term impacts, was a major concern. With experience, children can be allocated to the appropriate KOSCHI category within a matter of minutes.
The scale places a high emphasis on concentration, behaviour, and disinhibition, which are common problems in this group of children. Disinhibition was described by Rutter12 as the behavioural outcome most typical of acquired brain injury. It is highly socially disabling, and results in continuing dependency even in the absence of physical sequelae. Injuries to the frontal lobes of the brain are common after TBI, although they may not manifest until puberty.13 There is increasing awareness of the long term hidden effects of this type of damage to immature brains.14-16 For example, a child who has been allowed to return to mainstream school and apparently “fully recovered” (category 5a or 5b) may start to fail as educational expectations and demands increase, the impact of the injury on new learning17 ,18 becomes apparent, and he slips back into category 4b. The development of secondary behaviour problems may contribute to this decline in school achievement.19
There is inevitably some potential for ambiguity in such a simple instrument. For example, a child with minor head injury who has a partial spinal cord transection will acquire a functional disability which could place him in category 4a, irrespective of complete recovery from the brain injury per se. Where possible, information on the sequelae of the child's brain injury rather than of other injuries, should be used. A much more common problem encountered is the child with premorbid learning and/or behaviour problems and as a general rule, assignment to a category should be based on change in function following TBI. Such ambiguities will be less relevant to summary outcome data for large groups.
Although the scale was originally designed for children aged 2 to 16 years of age, there is no reason why it should not be used under the age of 2 years with the caveat that problems in this young age group are commonly underestimated. Falls and intentional shaking injuries are important causes of TBI in this group. Even if the child scores normally at this young age it must be borne in mind that problems such as dyspraxia or language disorders may emerge with time, and the child may regress through the categories as the path of development becomes more aberrant. This is particularly true of infants with subdural haemorrhages,20 in whom shaking causes widespread axonal shearing.21
The KOSCHI is a short hand description of the child's level of functioning at a point in time. As such it provides a useful clinical tool for documenting recovery in the individual child, as well as for monitoring the burden of disability caused by TBI, for service planning, and for evaluating rehabilitation programmes and the effects of service and research intervention. We hope that, like the GOS, it will be further validated by comparison with outcome predictors such as the Glasgow Coma Score and measurements of post-traumatic amnesia.
Acknowledgments
Thelma Colaco was supported by a grant from the Joint Research Committee. We thank S Beljung for preparation of the typescript, C Hawley, H Greenberg, E Jones and C Pickthall for their participation in the inter-rater reliability exercises, and M Aitken for statistical advice.