METHODS

Detailed information on the sample design and data collection has been reported elsewhere.¹⁴

Sampling strategy

Medical records were drawn from a representative sample of 13 public hospitals selected from 20 institutions with 100 or more beds. Sampling followed stratification by hospital type and geographical area across New Zealand. The national sample comprised all six large tertiary service facilities; a probability proportional to size (PPS) sample of four smaller secondary service facilities with more than 300 beds; and a PPS sample of three secondary service facilities with less than 300 beds.

The survey population was defined as all patient admissions for the calendar year 1998. Exclusions followed those defined for the Harvard Medical Practice Study.¹ Day and rehabilitation only cases were excluded because adverse events were highly unlikely to occur and would, in any case, require acute inpatient admission or transfer for any significant treatment—at which point they would enter the sampling frame and become eligible for selection. Psychiatric cases were excluded because they were fundamentally different and the study instruments were not designed to process them.

The sampling frame for each hospital was a list of all eligible admissions in that hospital. With the target of 500 cases per hospital (to reach statistical power requirements), and to allow for non-location, a systematic list sample of 575 admissions was selected from each of these hospitals for the year 1998, with cases ordered by admission date. The full medical record associated with each sampled admission was analysed for the occurrence of an adverse event, either in a public hospital or other setting. To be included in the analysis an adverse event had to have been detectable within the sampled admission.

Record review

The core data collection procedure of the study was a two stage retrospective review of a representative sample of medical records from each selected hospital. This two stage procedure and the review forms directly replicated in all important respects the US and Australian studies.^1,4

The first stage was the screen undertaken by trained registered nurses (RN). The purpose of this stage was to ascertain if the hospitalisation in question—the sampled admission—met any of 18 screening criteria selected as potentially indicative of an adverse event such as unplanned re-admission or return to operating theatre. The second stage undertaken by specifically trained and highly experienced physicians (MO) used an instrument relying on structured implicit review—that is, the guided exercise of professional judgement. The objective of this exercise was to determine whether an adverse event was detectable within the sampled admission and, if so, to characterise its causation, preventability, and impact on length of stay and patient morbidity.

It should be noted that the documentation in sampled medical records was sufficiently detailed and comprehensive to permit full completion of study instruments, and there was evidence of internal consistency in the data on key study variables—for example, the relationship between assessed patient disability and extra hospital workload. Levels of reliability of reviewer (MO) judgements were moderate (87.5% agreement (kappa 0.47) with an expert reviewer (ER) on adverse event (AE) determination in a 1 in 10 subsample of cases), but were within established norms for comparable studies internationally.¹⁴ Other data are as follows: RN/MO on screening criteria presence (95.5% agreement, no kappa possible); RN/MO on AE presence (81.6% agreement, kappa 0.47); RN/ER on criteria presence (70.7% agreement, kappa 0.42); RN/ER on AE presence (83.7% agreement, kappa 0.35).¹⁴

Definition of variables

To qualify as an adverse event for this analysis an incident had first to have been recorded in the patient notes as part of the standard “narrative” report of clinical activities by a healthcare professional during the sampled admission (although the incident could have occurred before and precipitated the admission). This incident had then to be detected in the clinical narrative, assessed, and deemed to qualify as an adverse event by a study physician reviewer applying a standard protocol. An adverse event was operationally defined as an unintended injury resulting in disability and caused by healthcare management rather than the underlying disease process. Some examples are given in box 1, ranging from an instance that did not qualify as an adverse event, through a case occurring outside hospital, to others taking place in a hospital setting.

Preventability of an adverse event was assessed as an error in healthcare management due to failure to follow accepted practice at an individual or system level. Disability refers to impairment of physical or mental function and/or prolonged hospital stay, as reported in the medical record. Serious impact of an adverse event was defined as permanent disability (lasting more than 1 year) or death. The examples outlined in box 1 are of low and high preventability events of varying patient impact (low to moderate).

Adverse events were categorised according to broad clinical classifications outlined in box 2. They were also assessed for systems failure.

The potential for prevention of recurrence of particular adverse events was assessed by physician reviewers identifying broad “areas of effort”. These are outlined in the lower part of box 2.

Hospital patient factors were age, sex, ethnicity (European, Maori, Pacific, other), area deprivation score (NZDep96 decile, an area based index of social deprivation derived from patient domicile code),¹⁵ and principal diagnosis (25 major diagnostic categories derived from Australian AN-DRG 3.1).¹⁶

Statistical analysis

The first objective of the investigation was to establish the pattern of medical injury among patients in New Zealand public hospitals using the key outcome measures (occurrence, preventability, and patient impact of adverse events). These results are presented using rates, percentage distributions, means, odds ratios, and associated confidence intervals, all adjusted to account for the stratified cluster sample design. The cases were weighted to account for unequal selection probabilities. Each hospital was given a weight inversely proportional to its selection probability. Variance estimates were adjusted for three strata which were defined by hospital bed size, and for the clusters as represented by the 13 hospitals.¹⁷

The second objective was to assess the level of serious preventable patient harm in relation to results from published studies conducted in tort jurisdictions. This calls for careful calculation of rates for particular study subgroups in order to achieve direct comparability with other investigations.

The odds ratios were estimated using multiple logistic regression with adverse event occurrence as the binary dependent variable. Predictors were patient factors as follows: age (30–64, 65+, reference = 0–29 years); sex (male, reference = female); ethnicity (Maori, Pacific, reference = European/other); area deprivation score (high (deciles 6–10), reference = low (deciles 1–5)); and case mix according to principal diagnosis (newborn/neonates, pregnancy/childbirth, nervous system, circulatory system, respiratory system, digestive system, musculoskeletal system, reference = remaining categories each comprising less than 5% of all admissions). The adjusted odds ratios show the effect of each patient factor, controlling for the effects of the other patient factors in the model. Appropriate variance estimates were calculated.

RESULTS

Pattern of medical injury

Among 6579 records reviewed, 735 incidents recorded in patient notes by a healthcare professional during the hospital episode associated with the sampled admission were classified by study reviewers as adverse events (an incidence rate of 11.3 per 100 admissions). Some key details of the events are shown in table 1. One fifth of the events occurred outside hospital, over 60% were judged preventable, approximately 15% were associated with death or permanent disability and, on average, an extra 9.5 days’ stay in hospital resulted. After excluding both events occurring outside public hospitals and those not considered preventable, the rate reduced by approximately half (to 5.2 per 100). Restricting events further to the most serious reduced the rate to less than 1 per 100 with a higher impact on hospital workload of just under 4 weeks.

The focus of the current investigation is on preventable in-hospital events, and these are presented in table 2 according to clinical category. The rate of occurrence of such events was just over 5%, with about half having some degree of system involvement (one third of all “mentions”). Drug events accounted for less than 10% of the total.

An important consideration is the potential role that patient attributes may play in influencing the pattern of medical injury, and this is addressed in table 3 with adjusted odds ratios for potential patient predictors of preventable in-hospital adverse events. Older patients, patients of Maori and Pacific ethnicity, and those admitted for musculoskeletal disorders were at greater risk of a preventable event.

Table 4 shows those events that had a significant patient impact. Such events are associated with just under 1% of admissions. The potential for the prevention of these events was assessed and a high proportion of events was seen to have some element of system involvement, with considerable contributions also from consultation and education.

Serious patient harm

This study sought to assess whether the level of serious preventable in-hospital medical injury in a well established “no fault” jurisdiction like New Zealand is within the range reported for similar studies under tort. Table 5 shows the figures from comparable studies in Australia⁴ and the US,^18,19 identifying the closest definition to the one adopted in table 4. All the US rates were less than 1% and those from Australia were less than 2%.

DISCUSSION

Key findings

Just over 5% of admissions to New Zealand public hospitals were associated with a preventable in-hospital adverse event, half of which had a significant system involvement. Older patients, those with an ethnic minority background, and certain clinical conditions were disproportionately affected. In analysing the subgroup of serious events the reviewers identified protocol development and implementation as an area for preventive activity in about one quarter of cases, with a similar figure for communication. Consultation with colleagues—another aspect of communication—was the largest single category, with education not far behind.

When assessed against directly comparable rates from tort jurisdictions, the rate of serious preventable patient harm for New Zealand (just under 1 in 100) was close to the published results from the US and Australia, suggesting that the underlying level of serious risk to patient safety is relatively uniform across medicolegal systems.

Strengths and limitations

The principal strength of the study is its application of a standard audit style protocol to medical records for a representative cross section of the New Zealand public hospital patient population.¹⁴ A further advantage is that the protocol and two stage review process are direct replicates from comparable studies across a range of countries.^1–5

However, there are also a number of potential shortcomings in a study of this kind, including the reliance on medical records (only a partial insight into the clinical setting), the retrospective nature of the methodology (difficult to reconstruct events), and issues about the reliability and validity of reviewer judgements (generally moderate).²⁰ Indeed, some investigators have gone so far as to call into question the underlying methodology.²¹ While these shortcomings undoubtedly weaken the impact of studies of this kind, they do not necessarily vitiate the overall thrust of this investigation and its findings, which are grounded in the judgements of experienced clinicians following a structured and standardised protocol. They point instead to the need for a multiplicity of approaches in research on medical injury and the quality of care; audit style retrospective record review clearly only provides one “piece of the jigsaw”.²²

Interpretation and implications

Little research has been published to date on medical injury in “no fault” medicolegal jurisdictions. The current investigation reveals a particular pattern in the occurrence, preventability, and impact of in-hospital adverse events. To the extent that comparisons are possible in similar studies, this pattern is much the same as that revealed in tort jurisdictions.^23,24

Questions still remain, however, as to whether the abolition of negligence litigation affects the quality of care and levels of patient safety. The rate of serious preventable patient harm reported in this investigation seems to be within the range of that published for studies under tort, indicating that medicolegal philosophy makes little difference to reported levels for these measures of quality and patient safety outcomes. The medicolegal environment may thus be a necessary but not a sufficient condition, suggesting that greater focus is required on the more immediate management context of clinical practice.²⁵

Controversies and future directions

There is considerable interest in the issue of patient safety and its compensation; the Bristol Royal Infirmary Inquiry has recommended the abolition of the system of clinical negligence,²⁶ and the National Health Service has now established a National Patient Safety Agency²⁷ and reported that litigation for medical negligence may amount to £2.6 billion in the UK.²⁸ Furthermore, doctors in many countries believe they are discouraged from reporting—or are not encouraged to report—medical errors.²⁹

The results of the current investigation do not rule decisively for or against “no fault”, at least as judged in the context of quality and patient safety issues. Future research should therefore consider alternative methodologies to counter the weaknesses of the audit approach, and should address more directly the issue of the potential for disclosure and error discussion under the “no fault” system.³⁰ Furthermore, the organisational context for quality needs to be incorporated into such research.²⁵

CONCLUSIONS

Patient risk of serious preventable medical injury at just under 1% appears to be no higher under the New Zealand system of “no fault” compensation for patients than in the medicolegal systems of Australia and the US governed by tort. The impact on hospital workload of these adverse events at an average of about 4 weeks, and the extent of system involvement in their causation at approximately 50%, highlight both the potential and the incentive for quality improvement.