Abstract
Background: Comprehensive systems for surveilling prescription opioid–related harms provide clear evidence that deaths from prescription opioids have increased dramatically in the United States. However, these harms are not systematically monitored in Canada. In light of a growing public health crisis, accessible, nationwide data sources to examine prescription opioid–related harms in Canada are needed. We sought to examine the performance of 5 algorithms to identify prescription opioid–related deaths from vital statistics data against data abstracted from the Office of the Chief Coroner of Ontario as a gold standard.
Methods: We identified all prescription opioid–related deaths from Ontario coroners’ data that occurred between Jan. 31, 2003, and Dec. 31, 2010. We then used 5 different algorithms to identify prescription opioid–related deaths from vital statistics death data in 2010. We selected the algorithm with the highest sensitivity and a positive predictive value of more than 80% as the optimal algorithm for identifying prescription opioid–related deaths.
Results: Four of the 5 algorithms had positive predictive values of more than 80%. The algorithm with the highest sensitivity (75%) in 2010 improved slightly in its predictive performance from 2003 to 2010.
Interpretation: In the absence of specific systems for monitoring prescription opioid–related deaths in Canada, readily available national vital statistics data can be used to study prescription opioid–related mortality with considerable accuracy. Despite some limitations, these data may facilitate the implementation of national surveillance and monitoring strategies.
The surveillance of prescription opioid–related deaths is a public health imperative.1 Over the past 2 decades, rates of prescription opioid–related death have increased rapidly in the United States2,3 and Australia.4,5 In the US, there is extensive public health surveillance of prescription opioid–related harms using multiple systems that collect information on deaths, emergency department visits, calls to poison control centres and drug seizures from law enforcement agencies.
Information related to prescription opioid–related harms in Canada is incomplete. These indicators are not systematically reported or monitored at national or provincial levels, and knowledge as to how these harms vary across regions in Canada is limited.6 Our knowledge of the prevalence of prescription opioid–related deaths is restricted to localized estimates obtained through various methods. National surveillance of prescription opioid–related deaths is essential to inform and evaluate solutions to this complex public health crisis in Canada.
Because individual review of postmortem toxicologic analyses is the gold-standard approach for identifying whether a death was related to prescription opioids, studying these deaths generally requires a review of coroners’ records. However, the process of abstracting prescription opioid–related deaths from coroners’ records is time and resource intensive. Furthermore, coroners’ records are not usually available to researchers.
An alternative source of data that captures cause of death in Canada is the Statistics Canada Vital Statistics Death Database. These data capture the contributing causes of death as determined by a physician, coroner or medical examiner and subsequently abstracted by professionals and coded using the International Statistical Classification of Diseases, 10th revision (ICD-10).7 Vital statistics data have the potential to provide researchers with national estimates of prescription opioid–related deaths, but the accuracy of ICD-10 codes is unknown. Therefore, the ability of these data to accurately identify prescription opioid–related deaths — which are often complex and have multiple contributing factors — is questionable.
Given the widespread availability of vital statistics data and the public health importance of conducting ongoing national surveillance of prescription opioid–related mortality, establishing the validity of algorithms designed to identify prescription opioid–related deaths in these data is important. We sought to validate several algorithms that use vital statistics data to identify prescription opioid–related deaths against data we abstracted from the Office of the Chrief Coroner of Ontario as a gold standard.
Methods
We conducted a retrospective validation study among Ontario residents who died of prescription opioid–related causes between 2003 and 2010. Residents from outside of the province who died in Ontario and Ontario residents who died outside of the province were not captured in our analysis.
This study was approved by the Research Ethics Board of Sunnybrook Health Sciences Centre, Toronto, Ontario.
Mortality data from the Office of the Chief Coroner for Ontario (gold standard)
We used data abstracted from the Office of the Chief Coroner for Ontario to identify all prescription opioid–related deaths in Ontario from Jan. 1, 2003, to Dec. 31, 2010. Canadian law requires that all unexpected or unnatural deaths be reported to the coroner so that cause or causes of death can be determined. Poisoning deaths would thus be among such cases reviewed by a coroner. In addition, coroners’ cases involve detailed toxicologic testing.
Using methods described in detail elsewhere,8 we defined the gold standard for identifying prescription opioid–related deaths as cases where the coroner’s report indicated either that prescription opioid concentrations were present at high enough levels to cause death or that a combination of drugs caused death that included one or more prescription opioids detected at a clinically significant concentration. Deaths from prescription opioids that had been dispensed from a pharmacy and deaths from prescription opioids obtained from diverted or nonmedical sources were classified as prescription opioid–related deaths. Deaths were not deemed opioid-related if opioids were merely present at therapeutic levels.
We identified prescription opioid–related deaths among all drug- or alcohol-related deaths that involved a prescription opioid from 2003 to 2010. A second abstractor independently reviewed a sample of the charts, and there was 100% agreement about the causes of death between the 2 abstractors. If the abstractor was uncertain about the cause of death, the case was reviewed by 1 or more physicians until consensus was achieved.
Mortality data from the vital statistics database
We used mortality data from the Statistics Canada Vital Statistics Database to identify individuals who died in Ontario from prescription opioid–related causes. The database captures all deaths of Canadian residents who die in Ontario, including information about a person’s cause or causes of death from the death certificate.7 Cause or causes of death on each certificate are coded using the ICD-10 system by mortality classification staff and entered into a computer database.9 A computer system selects the most important contributing cause of death for each record, termed the underlying cause of death, as per World Health Organization specifications. Additional information about the death is recorded in the multiple cause of death field. This field contains more detailed information about the death (e.g., the type of drug involved in a poisoning death) and lists other contributing causes of death. Each record lists 1 underlying cause of death and up to 11 multiple causes of death.
Definitions of prescription opioid–related death
We tested the validity of 5 algorithms designed to identify prescription opioid–related deaths in the vital statistics death data. These algorithms, which combine underlying and multiple causes of death coding to identify prescription opioid–related deaths, are presented in Table 1.
The first algorithm is based on definitions used by researchers at the US Centers for Disease Control and Prevention (CDC) to identify prescription opioid–related deaths using US mortality data.10,11 This algorithm specifies an underlying cause of death related to poisoning, and multiple cause of death codes that indicate at least 1 prescription opioid contributed to the poisoning death. This algorithm represents a strict definition of prescription opioid–related deaths, which could lead to the potential underreporting of deaths. Therefore, we tested 3 more inclusive variations of this algorithm. Specifically, algorithm 2 is similar to algorithm 1 but also captures deaths where the multiple cause of death is “other and unspecified narcotics” (code T406). Both algorithms 3 and 4 do not require the underlying cause of death to be poisoning. Instead, algorithm 3 only requires that the multiple cause of death codes indicate at least 1 prescription opioid contributed to the death. Algorithm 4 only requires that multiple cause of death codes indicate that either prescription opioids or other unspecified narcotics contributed to death. In efforts to further increase the sensitivity of these definitions, we developed algorithm 5 after reviewing the frequencies of ICD-10 codes assigned to individuals. Algorithm 5 builds on algorithm 4 by including any deaths identified as a narcotic poisoning in any cause of death field so long as there is no evidence of illicit narcotics (e.g., heroin).
Statistical analyses
We determined the best algorithm for identifying prescription opioid–related deaths in 2010 using vital statistics data by comparing the cohorts defined by the 5 algorithms to the cohort abstracted from the Ontario coroners’ data. We linked these data sets using unique encoded identifiers and analyzed them at the Institute for Clinical Evaluative Sciences. For each algorithm, we calculated the sensitivity, specificity, positive predictive value, negative predictive value, level of agreement (κ), area under the receiver operating characteristic curve (AUC), positive likelihood ratio and negative likelihood ratio. To identify the optimal algorithm, we selected the algorithm with the highest sensitivity and positive predictive value, with a minimum positive predictive value of at least 80%. These selection criteria are consistent with criteria used in similar studies.12 Upon identifying the optimal algorithm, we repeated the analysis by year (2003–2010) to determine whether the validity of the vital statistics data varied over time as the issue of prescription opioid–related deaths became better appreciated. We performed all analyses using SAS software (version 9.3).
Results
We identified 3549 prescription opioid–related deaths from coroners’ data, of which 3480 could be matched to records in the Satistics Canada Vital Statistics Death Database.
The performance of each algorithm in identifying prescription opioid–related deaths in the vital statistics data from 2010 is summarized in Table 2. The specificities and negative predictive values of the algorithms did not vary substantially, because deaths from prescription opioids represent a small proportion of deaths from all causes. Algorithm 5, which represents the most inclusive definition of prescription opioid–related deaths, had the highest sensitivity (89%). However, this algorithm had the lowest positive predictive value (70%), capturing a higher proportion of false positives when compared with the other algorithms. All of the remaining algorithms had a positive predictive value of more than 80%.
In terms of sensitivity, algorithms 2 and 4 performed equally well (75%) and outperformed algorithms 1 and 3. We selected algorithm 2 as the optimal definition of prescription opioid–related deaths because it had a higher positive predictive value (90%) than algorithm 4 (87%).
The performance of these algorithms in identifying prescription opioid–related deaths in previous years of vital statistics data (2003–2009) is detailed in Appendix 1 (available at www.cmaj.ca/lookup/suppl/doi:10.1503/cmaj.150349/-/DC1); however, we did not consider algorithm performance in years other than 2010 when selecting the optimal algorithm.
When using algorithm 2, although death data obtained from vital statistics consistently underestimated the number of deaths in the coroners’ data, the trends were similar over time (Figure 1). Algorithm 5 had the highest sensitivity but low positive predictive value, and therefore did not meet our criteria as the optimal definition for prescription opioid–related deaths. The use of this algorithm overestimated the number of prescription opioid–related deaths (Figure 1).
There was a slight improvement in the sensitivity of algorithm 2 from 2003 to 2010 (Table 3); with the exception of 2004, the positive predictive value was relatively constant over time.
Interpretation
This large, population-based study suggests that vital statistics data can be used, with some caution, to identify prescription opioid–related deaths when coroners’ data are unavailable.
Modifications of the CDC algorithm for identifying prescription opioid–related deaths from vital statistics data were able to identify three-quarters of true cases in Ontario in 2010. However, this underestimation still allows for meaningful examination of changes over time; with improving sensitivity as a result of more careful and detailed coding of deaths, the validity of vital statistics data will likely improve as well. In the absence of existing surveillance of prescription opioid–related harms in Canada, the finding that readily available national vital statistics data can be used to study prescription opioid–related deaths is of considerable public health and policy importance.
Owing to the paucity of data in this area, we could not identify other studies of opioid-related mortality in a Canadian setting. However, researchers in Finland have examined the accuracy of administrative mortality data in identifying illicit drug deaths, finding that 89% of opiate-related deaths based on positive drug findings were registered with opiate-specific ICD codes in national cause-of-death statistics.13 Similarly, US studies have found that direct analysis of death certificates can be used to detect 90% of cancer and 84% of coronary heart disease deaths but only 64% of deaths by stroke.14 The findings of our study suggest that the capacity of vital statistics data to accurately identify prescription opioid–related deaths is consistent with these estimates.
Strengths and limitations
This was a large, population-based study capturing all prescription opioid–related deaths over an 8-year period in Ontario, the most populous province in Canada (population 13.1 million in 2010). In addition, prescription opioid–related death data abstracted from coroners’ data and used as a gold standard was rigorously defined using coroners’ clinical findings and toxicological analyses and is likely a true representation of overdose deaths in this province. Finally, 4 of the 5 algorithms investigated in this study yielded very similar results, suggesting that these definitions for identifying prescription opioid–related deaths from vital statistics data are robust and would not easily be affected by small variations in coding.
Several limitations exist that must be considered if these data are to be used in the future. First, even the optimal algorithm to identify prescription opioid–related deaths in vital statistics data will underestimate the number of deaths. Second, changes in the accuracy of the algorithms in detecting prescription opioid–related deaths over time suggests underlying changes in coding practices or data quality. In particular, the improvement we saw in sensitivity may be a result of increased awareness of prescription opioid overdose as an emerging cause of death by either vital statistics personnel or coroners tasked with determining the cause of death. Indeed, the Ontario coroner’s office has incorporated prescription opioid–related deaths into training in recent years (Nav Persaud, University of Toronto, personal communication). However, it is not possible to rule out other causes of changes in coding practices or data quality. Finally, we were unable to obtain coroners’ data from another Canadian province, and our conclusions are based on the performance of these algorithms in a single database. The results of this validation study may not be generalizable to other jurisdictions, because coroners’ training, coding standards and information sharing can differ regionally. For this reason, we recommend that the accuracy of algorithms for identifying prescription opioid–related deaths from vital statistics data be tested in other jurisdictions.
Conclusion
Our results show that vital statistics data can be used to identify prescription opioid–related deaths in Canada with a considerable degree of accuracy. However, the validity of these definitions could be further improved by enhancing coding and information sharing to improve the quality and reliability of the data. Despite some limitations, the ability to use vital statistics data sets in this way can facilitate national surveillance and monitoring strategies that, until now, have been impossible owing to incomplete and inaccessible coroners’ data.
Vital statistics data could be used to show heterogeneity in prescription opioid–related deaths across geographic regions. Given the increasing rates of prescription opioid–related deaths in Ontario,8 and the great degree of interest in understanding this phenomenon at the national level, this has considerable implications for future research.
Prescription opioid–related deaths identified from coroners’ data and from the Statistics Canada Vital Statistics Death Database using algorithms 2 and 5 (see algorithm definitions in Table 1).
Description and technical definition of each of the algorithms tested
Performance of 5 algorithms in identifying prescription opioid–related deaths from the Statistics Canada Vital Statistics Death Database in 2010
Performance of algorithm 2 using data from 2003 to 2010
Acknowledgement
The authors thank the Office of the Chief Coroner of Ontario, which, as part of its public safety mandate, made available the relevant data on prescription opioid–related deaths in Ontario.
Footnotes
Competing interests: None declared.
This article has been peer reviewed.
Contributors: Emilie Gladstone, Kate Smolina, Steve Morgan and Tara Gomes conceived of and designed the study. Tara Gomes, Diana Martins, and Kimberly Fernandes were involved in the acquisition of data. Kimberly Fernandes conducted the data analysis. All of the authors were involved in the interpretation of the analysis. Emilie Gladstone drafted the manuscript, and all of the authors revised the work for important intellectual content. All of the authors read and approved the final version to be published and agree to be accountable for all aspects of the work.
Disclaimer: This study was supported by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the Ontario Ministry of Health and Long-Term Care (MOHLTC). The opinions, results and conclusions reported in this paper are those of the authors and are independent from the funding sources. No endorsement by ICES or the Ontario MOHLTC is intended or should be inferred.
Funding: This study was supported by the Ontario Drug Policy Research Network, which is funded by grants from the Ontario Ministry of Health and Long-Term Care (MOHLTC) Health System Research Fund; by the Institute for Clinical Evaluative Sciences (ICES), which is funded by an annual grant from the MOHLTC (to K.F., D.M. and T.G.); by a grant from the Canadian Institute for Health Research (CIHR) (to E.G., K.S., and S.M.); and by a CIHR Banting Postdoctoral fellowship (K.S.).
- Accepted October 6, 2015.
In this issue
Article tools
