Abstract
BACKGROUND: Evidence-based guidelines advise excluding pulmonary embolism (PE) diagnosis using d-dimer in patients with a lower probability of PE. Emergency physicians frequently order computed tomography (CT) pulmonary angiography without d-dimer testing or when d-dimer is negative, which exposes patients to more risk than benefit. Our objective was to develop a conceptual framework explaining emergency physicians’ test choices for PE.
METHODS: We conducted a qualitative study using in-depth interviews of emergency physicians in Canada. A nonmedical researcher conducted in-person interviews. Participants described how they would test simulated patients with symptoms of possible PE, answered a knowledge test and were interviewed on barriers to using evidence-based PE tests.
RESULTS: We interviewed 63 emergency physicians from 9 hospitals in 5 cities, across 3 provinces. We identified 8 domains: anxiety with PE, barriers to using the evidence (time, knowledge and patient), divergent views on evidence-based PE testing, inherent Wells score problems, the drive to obtain CT rather than to diagnose PE, gestalt estimation artificially inflating PE probability, subjective reasoning and cognitive biases supporting deviation from evidence-based tests and use of evidence-based testing to rule out PE in patients who are very unlikely to have PE. Choices for PE testing were influenced by the disease, environment, test qualities, physician and probability of PE.
INTERPRETATION: Analysis of structured interviews with emergency physicians provided a conceptual framework to explain how these physicians use tests for suspected PE. The data suggest 8 domains to address when implementing an evidence-based protocol to investigate PE.
Pulmonary embolism (PE) occurs when a blood clot lodges in the pulmonary arteries. If left untreated, the disorder can progress, causing worsening morbidity and may become fatal.1 Because of the acute nature of this condition, many patients with PE present to the emergency department.
Diagnosing and excluding PE using computed tomography pulmonary angiography (CTPA) alone can be problematic because of radiation exposure, anaphylaxis to contrast, misdiagnosis and “overdiagnosis” of inconsequential PE2 (leading to unnecessary anticoagulation therapy and psychological distress3). Choosing Wisely4,5 and the guideline from the American College of Physicians6 recommend the use of risk stratification tools, including the Pulmonary Embolism Rule-out Criteria (PERC) clinical decision rule,7 the Wells score8 and blood concentration of d-dimer. These tools use different predetermined diagnostic algorithms to indicate the need for CTPA.8–11 Evidence-based guidelines discourage further testing in patients at lower risk who have normal d-dimer levels, where imaging can cause more harm than benefit.12,13 However, many emergency physicians opt for CTPA as a stand-alone test for PE.14–17
It remains unclear why emergency physicians sometimes do not use validated diagnostic PE tools. Furthermore, implementation of computerized decision support systems has had little success in modifying this behaviour.18,19 We sought to develop a conceptual framework to describe how Canadian emergency physicians test for PE, and to document the cognitive and contextual barriers to using existing evidence-based diagnostic PE pathways.
Methods
Study design and population
This was a qualitative, 4-part interview study. We followed the COnsolidated Criteria for REporting Qualitative Research (COREQ) guideline for reporting. Participants were staff emergency physicians in Canada who were invited to an interview on “clinical decision-making.” We interviewed physicians from 3 hospitals in Hamilton to develop a provisional framework describing how emergency physicians test for PE, and why they choose the tests they use. We were mindful that Hamilton is the birthplace of Canadian thrombosis medicine, so we then interviewed emergency physicians from sites across Canada, until thematic sufficiency had been reached. The participating hospitals did not have computerized decision support for PE testing.
We sent an email invitation to every staff emergency physician working in the participating hospitals (9 hospitals in Hamilton, Ottawa, Montréal, Vancouver and Toronto). We used snowball sampling when there was an insufficient response to the email invitation.
Interviews
The structured interview was designed as a cognitive task analysis, a technique which elicits practitioners’ thinking around parts of their work that require thought, planning and action. The interview was piloted and refined before implementation. Appendix 1 (available at www.cmaj.ca/lookup/doi/10.1503/cmaj.201639/tab-related-content) includes the interview, links to the videos, an example mind map and the questionnaire. In phase 1 of the interview, the participant recalled a patient they tested for PE and explained their diagnostic approach and reasoning. In phase 2, the participant watched 2 videos of patients with possible PE symptoms in a simulation (the videos were chosen randomly from a selection of 4 videos). Participants were then asked to explain to the interviewer how they would test the patient (step by step) by drawing a mind map. In phase 3, they completed a questionnaire on their knowledge of the Wells score,8 age-adjusted d-dimer9 and the PERC rule.7 The interview finished with open-ended questions about barriers to using the diagnostic guidelines for PE (phase 4). All interviews were identical with the exception that participants from Hamilton reviewed an additional video of a patient with possible deep vein thrombosis. Our study analyzed only the PE-related cases. The interviews were conducted at each hospital in a private room. Our investigators trained the site research staff who did not have previous qualitative research experience and were either research assistants or undergraduate students with no previous relation to the participants. Interviews were audio-recorded and later transcribed.
Qualitative analysis
We used a constructivist grounded theory technique20 to develop a conceptual model that explained how emergency physicians choose PE testing. The qualitative analysis was performed by an investigatory team comprising an emergency physician researcher specializing in knowledge translation and education (T.M.C), an emergency and thrombosis physician who specializes in diagnostic PE research (K.d.W), a nonphysician expert in physician practice variation research (M.M.) and an undergraduate student with no previous experience of PE (S.Z.). The investigatory team met to discuss their inherent stances and assumptions to ensure reflexivity before the start of the study.
For the interviews involving participants in Hamilton, each interview was coded separately by 3 researchers (K.d.W., T.M.C. and M.M.) using a constant comparative approach.21 An iterative process was used whereby assigned codes were reviewed by the research team, who developed and refined a common code book. The team met on 7 occasions. We assigned similar codes to common themes to create our initial framework. The interviews were analyzed while interviews were ongoing. We ceased the interviews when no new themes emerged.
To refine this initial framework, 2 investigators (S.Z. and K.d.W.) coded the transcripts and mind maps from interviews conducted in 6 additional emergency departments, using the previous framework codes. The researchers met a total of 6 times. Where the investigators were unable to identify a suitable code, they agreed on new codes that were mapped to pre-existing themes or a new theme was identified. Themes were grouped into domains for the final version of the framework.
Statistical analysis
For each clinical case discussed in the interviews, 2 researchers independently recorded which clinical decision rules were used for PE testing. Participant demographics and quantitative interview data were reported as medians and interquartile ranges (IQRs) or proportions.
Ethics approval
Approvals were obtained from the Research ethics boards for all participating hospitals.
Results
We conducted 63 interviews (participant demographics found in Table 1). Between 2015 and 2016, we interviewed 16 emergency physicians working at 1 of 3 hospitals in Hamilton, from which 6 themes were derived (Appendix 2, available at www.cmaj.ca/lookup/doi/10.1503/cmaj.201639/tab-related-content). An additional 47 physicians completed interviews in the refinement stage, which took place between 2017 and 2018. Our analysis yielded 7 new themes and the final framework included 8 domains, which are described in the following paragraphs (Table 2). Example quotes can be found in Table 3.
Anxiety with pulmonary embolism
Participants viewed PE as a fatal disorder and indicated that identifying dangerous diagnoses is a pivotal role for emergency physicians. Missing the diagnosis of PE implied failing in this role. Therefore, the decision to start or not to start testing for PE evoked anxiety. Some participants expressed anxiety about using d-dimer or decision rules because of a lack of confidence in their safety or because of tool complexity and the potential for error.
Barriers to using the evidence
The time of day and degree of crowding in the emergency department affected the use of evidence-based tools, especially d-dimer, which was cited as delaying the ordering of an inevitable CTPA. Eight participants did not know that the Wells score and d-dimer were validated PE tests. Some participants felt patients attended the emergency department with an expectation of CT scanning.
Divergent views on evidence-based pulmonary embolism testing
Participants either liked using evidence-based diagnostic PE tools or expressed a range of negative opinions about these tools. The views were often contradictory: for example, PE clinical decision rules were cited as both saving time and taking too long to use, reducing testing and causing unnecessary testing, helping with patient flow through the emergency department and hindering patient flow.
Inherent problems with the Wells score
The Wells score was singled out by participants as being problematic to use in the emergency department. Many disliked that the Wells score is an intermediary step in the diagnostic process (leading to d-dimer testing or CT). Several disliked the subjective allocation of points for “PE being the most likely diagnosis,” and participants would allocate these points liberally, based on the presence of PE risk factors rather than on the likelihood of there being an alternative cause for the symptoms. A recurring concern was that the score added to an emergency physician’s cognitive load. Several found it frustrating that there are many new ways to interpret this score.
Pulmonary embolism testing must include CT imaging
In many instances, the participant saw CTPA as inevitable when there was suspicion of PE. For some, deciding to test for PE was synonymous with ordering CTPA. Computed tomography was considered the most trusted test for PE by the participants. Computed tomography pulmonary angiography was frequently referred to as the “definitive test;” obtaining a CTPA removed the anxiety of missing PE, the anxiety around deciding whether to test for PE and the anxiety associated with trusting d-dimer and a clinical decision rule. The goal was to obtain CTPA rather than diagnose PE. By arranging a CT, they handed over responsibility for PE testing to the radiologist. No participant remarked on the possibility of false-positive or false-negative CT results. The CTPA report was seen as the radiologist’s domain and the CT results accepted without question.
Gestalt artificially inflates pulmonary embolism pretest probability
One of the strongest themes was working with gestalt. Several participants were so comfortable with their gestalt that they would trust gestalt over clinical decision rules, d-dimer level and imaging results (e.g., by ordering CTPA for a patient with a low Wells score and normal level of d-dimer). We observed that when a participant used gestalt, they commonly overestimated the pretest probability of PE. Use of gestalt frequently led to ordering CTPA. Gestalt was mostly stated as “high risk” (when the Wells score was low). High risk could refer to the patient having risk factors for developing PE (e.g., being treated for cancer or having a history of PE). The word “risk” was transferred from one term (risk factor) to another (high risk) without further thought. The term high risk was also used when participants felt anxious about PE. Other times risk referred to the potential for a hemodynamically unstable patient to die. Risk signalled low confidence in d-dimer to safely rule out PE or that d-dimer would be a nuisance rather than a help. Labelling the patient as high risk facilitated obtaining CT.
Subjective reasoning and cognitive bias
Almost all participants stated they would follow an evidence-based strategy. However, in some instances, the diagnostic process did not include clinical probability stratification or d-dimer, and the participant defaulted to ordering CT. Physician reasoning included the following: they believed (wrongly) that the patient was an exception to the use of the Wells score; the Wells score often differed from their own gestalt estimate of pretest probability; they guessed that the Wells score would be high or that d-dimer would be elevated; and they appeared to “inflate” the Wells score to avoid the use of d-dimer.
Clinical decision rules are used mainly to rule out pulmonary embolism
Evidence-based testing for PE appeared more likely to occur when the participant’s gestalt pretest probability of PE was low. Participants appreciated being able to document the PERC rule in this instance for medicolegal reasons.
We were unable to detect theme patterns according to training, years in practice, city, and academic versus community practice. Diverse opinions about using evidence-based PE testing were voiced by all groups. Table 4 summarizes the PE testing and knowledge test results.
Interpretation
Our qualitative exploration of how Canadian emergency physicians test for PE yielded important insights. We found physicians felt anxious about missing PE, knowing when to test for PE and using evidence-based PE tests. Physician knowledge, time pressure and patient expectations were barriers to using evidence-based testing, with divergent opinions about the evidence. We found practical problems with using the Wells score in the emergency department. There was a focus on obtaining CTPA rather than diagnosing PE. Together, these issues explained the preference for using gestalt estimation of pretest probability, which appeared to artificially inflate the probability of PE. Physicians used several cognitive strategies to perpetually choose CTPA over evidence-based testing, which was more often reserved for patients who were very unlikely to have PE.
Gestalt was used in 66% of the cases and was the most common subjective reasoning tool to justify ordering CTPA over d-dimer. A study involving consecutive patients presenting to the emergency department in France and Belgium found that a physician gestalt estimate of pretest probability performed similarly to the Wells score.22 We found physician gestalt was used in place of a clinical probability score; however, in our study gestalt was not a pretest probability gauge. Instead, participant gestalt statements signalled the need or urgency for CTPA. Stating a patient was at high risk for PE permitted the physician to bypass d-dimer or ignore a normal d-dimer result. Gestalt is a “workaround” in the era of clinical probability estimation. Evidence-based PE testing has changed physician vocabulary. At first glance, physicians appear to follow evidence-based guidelines, but in reality, this evidence has little impact on the choice of test.
Our findings concord with previous research. There are only 2 previous studies exploring emergency physician behaviour around PE testing.23,24 In addition to finding a reliance on gestalt, both studies also reported contrasting views on the value of evidence-based PE testing and noted that physician gestalt tended to exaggerate the likelihood that a patient had PE. Other common findings were a lack of knowledge of the Wells score and time pressure influenced test choice.
We have highlighted gaps between evidence-based PE tests, the people who use the tests and environment where the tests are used. We found that test choices were influenced by the disease (fear associated with PE), the emergency department environment (time pressures and cognitive load), the test qualities (Wells score complexity), the physician (view of evidence-based medicine) and probability of the disease (evidence-based tests used more often when PE seems unlikely). The premise that an emergency physician will make testing decisions purely based on the numerical probability of disease seems unlikely to hold true in light of our findings. Probst and colleagues25 described the same issue in relation to ordering head CT scans for patients with head injuries in the emergency department, citing patient, system and physician factors influencing each decision. A 2015 qualitative study involving patients in the emergency department with minor head injuries reported that anxiety, time pressure, knowledge and patient factors also influenced the decision to order head CT.26 To effect changes in patient care, future tests should respond to a need identified by an emergency physician and should be designed specifically for the environement of the emergency department.
Limitations
There were limitations in our study design. We do not know whether those physicians who participated in our interviews differed from those who did not. We studied interview transcripts rather than real-time clinical observations. We did not record participant gender and therefore cannot comment on its effect. Our initial framework was derived from interviews performed at the Hamilton sites (which have a prominent thrombosis service). We did not identify practice differences among Hamilton and other sites; however, it is possible that the unique environment in Hamilton could have subconsciously shaped our framework. Our premise was that evidence-based PE testing was superior to alternative testing strategies, but we did not measure patient outcomes and cannot use our study findings to support this premise.
Conclusion
Analysis of structured interviews with emergency physicians provides a conceptual framework to explain how these physicians use tests for suspected PE, and why they choose the tests they use. These findings have important implications for future implementation in guidelines and the development of new PE tests.
Acknowledgement
The authors thank Natasha Clayton and the EMeRGE research program for their help to transcribe the interviews.
Footnotes
↵* Dr. Clive Kearon died on June 3, 2020, during preparation of this manuscript for publication.
Competing interests: Teresa Chan has received stipends from McMaster University and ALiEM LLC, and funding from the PSI Foundation outside the submitted work. No other competing interests were declared.
This article has been peer reviewed.
Contributors: Kerstin de Wit conceived the study. Kerstin de Wit, Teresa Chan, Mathew Mercuri and Clive Kearon designed the study. Michelle Turcotte and Emily Grusko developed the interview methods. Analysis of data was conducted by Kerstin de Wit, Teresa Chan, Mathew Mercuri and Sahar Zarabi. Kerstin de Wit and Sahar Zarabi drafted the manuscript. All of the authors contributed to editing of the manuscript for important intellectual content, gave final approval of the version to be published and agreed to be accountable for all aspects of the work.
Funding: Kerstin de Wit received the PSI Graham Farquharson Knowledge Translation Fellowship 2017–2020 and an Early Career Award from the Hamilton Health Sciences Foundation to fund the work of this study. Funding was also received from the McMaster Continuing Health Science Education Program Research and Innovation Fund.
Data sharing: Study coding is available on request to Kerstin de Wit (dewitk{at}mcmaster.ca).
- Accepted October 25, 2020.
This is an Open Access article distributed in accordance with the terms of the Creative Commons Attribution (CC BY-NC-ND 4.0) licence, which permits use, distribution and reproduction in any medium, provided that the original publication is properly cited, the use is noncommercial (i.e., research or educational use), and no modifications or adaptations are made. See: https://creativecommons.org/licenses/by-nc-nd/4.0/