METHODS

Data extraction

An electronic search of published reports was undertaken to identify studies published between January 1975 and April 2001 in English language journals. A computer generated Medline search was applied, using the terms “cardiac evaluation”, “cardiac risk”, “perioperative myocardial ischaemia”, “perioperative cardiac morbidity”, “myocardial infarction”, “perioperative outcome predictors”, and “major vascular surgery”, in conjunction with one of the following non-invasive tests used for detection of myocardial ischaemia: “exercise electrocardiography”, “continuous ambulatory electrocardiography monitoring”, “radionuclide ventriculography”, “dobutamine and dipyridamole stress echocardiography”, and “myocardial perfusion scintigraphy”. Additional references were obtained from the bibliographies of review articles and original papers.

Studies were included if perioperative (30 day) data on cardiac death and non-fatal myocardial infarction or the composite were reported, and if the absolute numbers of true positive, false negative, true negative, and false positive observations were available (including positivity thresholds), or were derivable from the data presented. If several studies were done on overlapping patient populations then one report was selected which had the largest sample size. If several tests were studied simultaneously, data from each were extracted separately. Studies in which preoperative coronary revascularisation occurred as a result of a positive test result were only included if patients who underwent such procedures could be excluded or analysed separately.

Pertinent data from the selected studies were extracted independently by two of us (MDK and EB), using standardised spreadsheets. Discrepancies were resolved by consensus. Information extracted included reference data (first author, journal, institution), publication year, number of patients, mean age, proportion of male patients, type of vascular surgery, percentage of patients with a history of coronary artery disease (defined as either past or current angina pectoris, history of myocardial infarction, or heart failure) and diabetes mellitus, type of radionuclide used, and the type of exercise performed or the type of pharmacological stress agent used. Criteria for positivity were recorded. This information is shown in table 1.

Data analysis

One hundred and thirty one studies published between January 1975 and April 2001 were screened. Fifty eight met the inclusion criteria (table 2). Data on some explanatory variables were not specified in the studies included. In seven studies^{14,15,18,22,37–39} the mean age was absent; in six^{14,15,24,36,37,42} the sex distribution was absent; in three^10,13,14 the proportion of patients with a history of coronary artery disease was not specified; and in 20^{7,10,13–16,20–23,25–31,35,37,40} the proportion of patients with diabetes was not given. Estimates for these variables were used, based on a best subset regression analysis, so that the maximum number of selected studies could be included. Weighted mean values for the missing data using sensitivity analysis or excluding studies from the analysis did not alter the results. Therefore, all selected studies were included for analyses.

Differences in baseline clinical characteristics between the study populations were evaluated using χ² statistics. To account for a possible source of heterogeneity in diagnostic threshold between studies, pooled results weighted by the sample size of each study were calculated using a random effect model, based on a single treatment effect and standard error for each of a set of studies.⁶² Results are presented as percentage sensitivity and specificity with 95% confidence intervals (CI).

Summary receiver operating characteristic (SROC) curves were generated to describe diagnostic performance over a range of threshold values for each non-invasive test (see the appendix). Univariable and multivariable regression analyses were undertaken to study the influence of clinical and study characteristics on test performance, including the number of patients tested and operated on, the mean age of the patients, the proportion of men, the proportion of patients with a history of coronary artery disease, the proportion of patients with diabetes mellitus, and the year of publication.

Comparisons using SROC analysis were also undertaken to enable us to study diagnostic performance between separate tests. In each case we included all significant explanatory variables, along with the variable indicating the test comparison. We developed models with identical explanatory variables across all comparisons. The differences in diagnostic performance between separate tests are represented by the relative diagnostic odds ratios with 95% CI. The relative diagnostic odds ratio indicates the diagnostic performance of a test, with a value larger than 1 indicating better discriminatory power, a value equal to 1 indicating no difference, and values below 1 indicating reduced discriminatory ability. In order to adjust for the fact that multiple comparisons were made, a probability value of p ⩽0.01 was considered significant. All statistical analyses were done using “meta” and “metareg” commands for STATA 6.0 for Windows (STATA Corporation, Texas, USA).

RESULTS

Clinical and study characteristics

A summary of the clinical characteristics of the studies included in the meta-analysis is given in table 3. Mean age was similar between the studies. The majority of patients in the studies were male, with no significant difference between the studies (p=0.380). Coronary artery disease was more common in patients who underwent radionuclide ventriculography, ambulatory ECG, and myocardial perfusion scintigraphy (p=0.03) compared with the other tests. The prevalence of diabetes was less than that of coronary artery disease, with no significant difference between the studies (p=0.06). Vascular surgery was not cancelled because of a positive test result after ambulatory ECG, radionuclide ventriculography, or dobutamine stress echocardiography. Preoperative revascularisation was undertaken following a positive test result in 16 patients who underwent exercise ECG, in 70 who underwent myocardial perfusion scintigraphy, and in 12 who underwent dipyridamole stress echocardiography. Patients undergoing preoperative coronary revascularisation were excluded or analysed separately in these studies. No operations were cancelled as a result of any exercise ECG abnormalities, but in 36 cases the operation was cancelled after a positive test result during myocardial perfusion scintigraphy.

Weighted pooled results

The diagnostic test performance for individual studies is outlined in table 3. In pooled data weighted by the number of patients with and without disease in each study, dobutamine stress echocardiography showed the highest sensitivity (true positive ratio) of 85% (95% CI 74% to 97%) with a specificity (1 – false positive ratio) of 70% (95% CI 62% to 79%) compared with the other tests (table 3).

Summary receiver operation characteristic analysis for each diagnostic test

In a univariable analysis for ambulatory ECG, none of the selected clinical risk factors was a significant predictor of the performance of the test. In fig 1, the SROC curve describes the test characteristics of ambulatory ECG monitoring in the studies included. When the selected clinical risk factors and study characteristics were tested in univariable analyses, none of the characteristics changed the SROC curve substantially. This was also observed when the performance of exercise ECG and dobutamine stress echocardiography was tested in separate univariable analyses—again analyses of exercise ECG and dobutamine stress echocardiography showed that none of the clinical and study characteristics changed the SROC curves (fig 1). Unlike the above described non-invasive tests, separate univariable SROC analyses for radionuclide ventriculography and myocardial perfusion scintigraphy showed that among the clinical risk factors and study characteristics only publication year changed the SROC curves significantly. The performance of the tests diminished with a later year of publication (fig 1). Remarkably, the estimates of sensitivity (true positive rates) and 1 – specificity (false positive rates) for dipyridamole stress echocardiography were inversely correlated when individual studies were plotted (fig 1). In this case only pooled sensitivity and specificity could be calculated. Thus the SROC analysis was not done and an SROC curve for dipyridamole stress echocardiography was not constructed.

• Download figure
• Open in new tab
• Download powerpoint

Figure 1

Graphs showing summary receiver operating characteristic (SROC) curves for ambulatory ECG, exercise ECG, dobutamine stress echocardiography, radionuclide ventriculography, and myocardial perfusion scintigraphy. The horizontal axis represents the false positive rate (1 – specificity) and the vertical axis the true positive rate (sensitivity). The graph for dipyridamole stress echocardiography represents a plot of the estimates of the true positive rate for dipyridamole stress echocardiography against the estimates of false positive rate, with a solid line representing the regression line. In all graphs, solid circles represent studies with more than 100 patients and open circles represent studies with less than 100 patients.

Comparison of diagnostic tests

Table 4 shows the results of the comparison of SROC analyses with (table 4B) and without (table 4A) adjustment for publication year. In the present study only publication year was a significant predictor of test performance for some of the tests analysed (radionuclide ventriculography and myocardial perfusion scintigraphy). In order to compare differences in diagnostic performance between the studies, the variable “publication year” was also included in all comparisons.

Ambulatory ECG performed no better than exercise ECG or myocardial perfusion imaging. Although there was a trend for ambulatory ECG to have a lower predictive performance than radionuclide ventriculography (p = 0.04) or dobutamine stress echocardiography (p = 0.03), this did not reach significance in univariable analysis and after correcting for publication year. Indeed, after adjustment for publication year an inverse relation was observed between radionuclide ventriculography and ambulatory ECG, though this was not significant. Exercise ECG showed a trend for a better discriminative power than ambulatory ECG and myocardial perfusion scintigraphy, and a reduced discriminative ability compared with radionuclide ventriculography and dobutamine stress echocardiography. However, these differences were non-significant (table 4). Myocardial perfusion scintigraphy showed lower discriminatory ability than ambulatory ECG, exercise ECG, or radionuclide ventriculography, though the differences were also non-significant. However, myocardial perfusion scintigraphy had a substantially lower discriminatory power than dobutamine stress echocardiography (p = 0.001), and this difference remained significant after adjusting for publication year (p = 0.002). Finally, dobutamine stress echocardiography showed a positive trend towards a better diagnostic performance than all the other tests, but this only reached significance in comparison with myocardial perfusion scintigraphy.

DISCUSSION

Our report is a meta-analysis of contemporary papers on six non-invasive tests used for preoperative risk stratification in patients selected for vascular surgery. We used an innovative meta-analytic method to estimate the diagnostic accuracy of these tests from multiple studies. The accuracy of the tests is presented and compared using a summary ROC curve, and the performance of individual tests was corrected for selected patient and study characteristics. Our results show that pharmacological stress tests have a higher overall sensitivity and specificity than the other tests. In particular, dobutamine stress echocardiography showed a positive trend for better diagnostic performance than the other tests, but this was only significant in comparison with myocardial perfusion scintigraphy. Ambulatory ECG, exercise ECG, and radionuclide ventriculography yielded a lower sensitivity and reasonable specificity, but no significant difference in predictive performance.

Ambulatory ECG monitoring showed low sensitivity and higher specificity in the present study, but no significant difference in predictive performance. The use of ambulatory ECG monitoring for perioperative cardiac risk assessment was first described by Raby and colleagues.⁶ They reported a sensitivity of 75% and a specificity of 83% for predicting cardiac death and non-fatal myocardial infarction. In later studies the predictive value of the test was corroborated, but the sensitivity was less than reported here.^9,10 Variation in end point composition, surgical procedures, and the timing of events could explain the observed differences in sensitivity and specificity between studies. The advantages of the test are that it is cheap and widely available. However, the presence of resting ECG changes (bundle branch block, left ventricular hypertrophy, digitalis use) may preclude reliable ST segment analysis in 40% of patients.⁶³ Hence, the combination of low sensitivity and resting ECG changes limits the application of the technique.

In our study the pooled data showed reasonable sensitivity and specificity for exercise ECG compared with the other tests. However, SROC analyses showed that it did not perform better than the other non-invasive tests. This observation may be explained by type II error—that is, differences could be missed owing to lack of power. Conventional exercise ECG is considered the most physiological form of stress.⁶⁴ However, the test may not always be feasible because of exercise intolerance in such patients.

The value of radionuclide ventriculography for predicting perioperative cardiac complications has been assessed in many studies.^20–27 A preoperatively assessed low ejection fraction (<35%) showed a relatively poor sensitivity but a high specificity in our meta-analysis. The SROC analysis showed that resting left ventricular dysfunction, as determined by radionuclide ventriculography, did not have a significantly better predictive performance than the other tests. The observed limitation of the predictive performance of this test may be explained by the failure to detect severe underlying coronary artery disease, changes in predictive value over time, and improved anaesthetic and surgical perioperative care. Thus, radionuclide ventriculography may not be a suitable test for preoperative risk stratification. However, a low ejection fraction may be a useful predictor of long term survival.⁶⁴

In patients unable to perform adequate exercise (and in most vascular surgery patients unable to exercise), a non-exercising test is mandatory. In this regard, myocardial perfusion scintigraphy—often combined with clinical risk assessment—is the most extensively studied non-invasive approach to cardiac risk stratification. In the present study a high sensitivity but a low specificity was observed for perioperative hard cardiac events, with a lower diagnostic accuracy than with dobutamine stress echocardiography. The earliest studies (between 1985 and 1987) showed that patients with fixed or reversible scintigraphic myocardial perfusion defects were at increased cardiac risk.^28,29 However, these results were later questioned by other investigators.^38,42 The poor prognostic value observed in these studies may reflect small sample sizes.

Comparison of myocardial perfusion scintigraphy with stress echocardiography in our study showed that dobutamine stress echocardiography performed significantly better. There are several possible explanations for these findings. First, myocardial perfusion scintigraphy is more widely used in consecutive patients presenting for vascular surgery than in selected patients with clinical risk factors; second, unblinded test results are available to clinicians, thus influencing perioperative care; third, repeat imaging 3–4 hours after thallium infusion may not allow sufficient time for thallium redistribution; and finally, thallium uptake may be uniformly restricted in patients with severe and diffuse coronary artery disease. Nevertheless myocardial perfusion scintigraphy is a valuable test for cardiac risk assessment, especially in patients with contraindications to stress echocardiography, with a reported complication rate of 3.9% in the studies included in the present meta-analysis. However, myocardial perfusion scintigraphy should be avoided in patients with significant bronchospasm, critical carotid disease, or on regular theophylline treatment.⁶⁴

Dipyridamole stress echocardiography has been proposed for cardiac risk stratification in patients undergoing major vascular surgery.^50–53 The low false positive rates and extremely high negative predictive values can make this test a useful predictor in low risk scenarios. Indeed, in the present meta-analysis dipyridamole stress echocardiography had a higher specificity than myocardial perfusion scintigraphy or dobutamine stress echocardiography. However, there have been limited numbers of studies published to date, and in these studies the true positive and false positive rates for dipyridamole stress echocardiography showed a negative correlation when they were plotted graphically. Hence only weighted pooled sensitivity and specificity could be calculated and the performance of the test could not be studied in further detail. The high specificity of dipyridamole stress echocardiography in clinical practice may indicate that the test can identify patients at less severe ischaemic responses. These patients may not need to undergo further testing and can proceed to major vascular surgery directly. However, in clinical practice it is more valuable to have a sensitive test that can identify patients at increased risk of perioperative cardiac events. Further conclusions about dipyridamole stress echocardiography are limited by the reported differences in sensitivity and specificity between the few studies reported to date.

Dobutamine stress echocardiography showed a similar high sensitivity, but a significantly higher specificity, compared with myocardial perfusion scintigraphy. Comparison using SROC analysis showed a trend towards better performance than the other tests, but this was only significant versus myocardial perfusion scintigraphy. As with dipyridamole, dobutamine stress echocardiography has been proposed for cardiac risk stratification owing to its high negative predictive value.^54–61 Moreover, in the present study this test showed significantly higher sensitivity than dipyridamole stress echocardiography. The role of dobutamine stress echocardiography in cardiac risk stratification has been studied extensively in recent reports.^59,61,65 The results of these studies suggest that the investigation can be done safely and with reasonable patient tolerance and may provide additional information about valvar dysfunction, in contrast to myocardial perfusion scintigraphy. The test has certain limitations—for example, it should not be used in patients with serious arrhythmias, severe hypertension, or hypotension. However, in a recent report of the use of dobutamine stress echocardiography in 6595 patients it was found that the incidence of cardiac arrhythmias and hypotension was 8% and 3%, respectively, and these complications were well tolerated and rarely required treatment.⁶⁶

Study limitations

The study has several limitations. Meta-analyses are subject to publication bias—that is, studies with a significant result are more likely to be submitted. Heterogeneity of study design is another aspect that may influence the interpretation of the results. The predictive value of a given test can be influenced by at least two important factors: patient selection and the blinding of test results. A test done in consecutive patients may have a lower diagnostic accuracy than the same test done in a selected group of patients. This is commonly referred to as selection bias and it occurs when consecutive cohorts of patients with a high prior probability of coronary artery disease, and therefore of adverse cardiac outcome after surgery, are more likely to get the test. The predictive value of a given test may also be influenced by the availability of the test result to the treating physicians. In this case patients with positive test results may undergo less invasive operations or receive better perioperative care, such as cardioprotective drugs, invasive haemodynamic monitoring, and admission to an intensive care unit. Studies in which patients are enrolled consecutively and treating physicians are blinded to test results are more likely to provide a relatively unbiased estimate for a given test.

Conclusions

Our meta-analysis shows that ambulatory ECG monitoring has a relatively low sensitivity and a low specificity, with no incremental value over the other tests. Furthermore, resting ECG changes frequently preclude reliable assessment of the ambulatory ECG and this test is therefore not recommended for perioperative risk assessment. Radionuclide ventriculography had the highest specificity but a relatively low sensitivity, with a limited predictive performance for perioperative events. This test should not therefore be considered as a tool for preoperative cardiac risk assessment. The test of choice in most ambulatory patients is an exercise ECG done according to the American College of Cardiology/American Heart Association guideline.⁶⁴ However, most vascular surgical candidates have important abnormalities on their resting ECG and are unable to perform adequate exercise. In such patients stress echocardiography or myocardial perfusion scintigraphy should be considered. In the current study dobutamine stress echocardiography showed a similar sensitivity to myocardial perfusion scintigraphy but a higher specificity and a better overall predictive performance. Moreover, dobutamine stress echocardiography is the favoured test if there is an additional question of valvar or left ventricular dysfunction. Dipyridamole stress echocardiography had a lower sensitivity and a higher specificity than myocardial perfusion scintigraphy. However, further conclusions about dipyridamole stress echocardiography are limited by the reported differences between studies and the limited number of studies reported to date.

Meta-analysis of six non-invasive tests showed a positive trend for dobutamine stress echocardiography to have a better diagnostic performance than the other tests, but this only reached significant difference in comparison with myocardial perfusion scintigraphy.