Computer support for determining drug dose: systematic review and meta-analysis

Inclusion criteria

We identified all comparative studies in which computers were used to help determine the most appropriate drug dose. The criteria for entry into the review were standard for reviews undertaken by the Cochrane Collaboration on Effective Professional Practice and include methodological and quality criteria for rigorous design of experimental and quasi-experimental studies.4

Methodological criteria were

• Studies using any objective measure of patient outcome or provider behaviour, randomised or quasi-randomised by patient, doctor, practice, or provider of health care

• Interrupted time series with a clearly defined intervention and at least three time points before and three after the intervention

• Non-randomised studies controlled at a second site with data before and after the intervention and appropriate choice of control site.

We included all studies using a reliable, objective, predetermined measure of the process or outcome of health care. This includes studies comparing computer aided decisions either to unassisted decisions or to decisions made using aids such as nomograms, as well as studies in which the computer directly administered the drug to patients (such as with a computer controlled pump). We excluded studies in which the computer simply suggested giving or withholding a drug. The criteria were applied independently by two researchers and any disagreements were resolved by group discussion.

Search strategy

Relevant studies were located from the specialised register of studies of the Cochrane Collaboration on Effective Professional Practice.5 This register is updated by electronic searches and hand searches of relevant journals. We also located references through bibliographies of related topics and contact with experts and pharmaceutical companies. We made specific searches of Medline and Embase from 1966 to June 1996 to identify relevant references. The search terms were “computer assisted decision making” or(prescr* and comput*) and (“randomised controlled trial” or “random allocation” or “double blind method”). Search strategies were modifications of those designed to give a high yield of randomised controlled clinical trials.6 In addition, we hand searched issues of Therapeutic Drug Monitoring published from January 1993 to July 1996.

Outcomes

Outcome measures, determined in advance, were

• Proportion of patients in which drug dose is changed because of computer advice

• Proportion of patients with unwanted effects of treatment

• Proportion of patients with blood concentrations of drug or a physiological measurement within the desired range

• Differences in blood concentrations of drug or physiological measurements attributable to computer support

• Time to achieve therapeutic control

• Proportion of patients with improved outcome attributable to computer advice.

To these we subsequently added changes in the cost of treatment attributable to computer support.

Data extraction

Two researchers reviewed each study independently and, using a standard form, extracted data on methodology, outcomes, and quality criteria.4 We recorded the unit of allocation and analysis, concealment of allocation, blinding, statistical power, follow up of patients and professionals, baseline measurements, and protection of the control group from contamination by the intervention. We calculated the mean difference in outcome with computer support, as a percentage of the mean value without support for all outcomes fitting our criteria for inclusion. These relative percentage differences were used in the narrative part of the review.

Quantitative analysis

Studies with comparable outcomes were divided into groups for meta-analysis according to outcome measure. Of the six predefined outcome measures, only four provided suitable data for meta-analysis: dose of drug; blood concentrations of drug; time to reach therapeutic concentration or effect; and difference between patients' drug concentration and target concentration. Four separate meta-analyses were performed.

We estimated effect sizes as standardised weighted mean differences for each outcome in each study where the relevant data were available. The effect size is a statistical measure of the impact of the intervention that is independent of the units used to measure an outcome. This measure quantifies the effect of an intervention in units of standard deviation and allows comparison of studies of the same intervention that measured different outcomes.

We used a random effects model to combine the effect sizes to give an overall effect for each subgroup of studies. This model was chosen because the outcomes we combined were for studies on different drugs and different diseases. The random effects model allows quantitative combination of outcomes but does not assume that all interventions have the same underlying effect. If the outcome was measured at different times in the same study, we selected the value nearest the midpoint of the intervention period. When there were related outcomes from the same study we used the mean of the effect sizes.7 In this way only a single effect size for each study was pooled. We performed calculations using the RevMan software provided by the Cochrane Collaboration.

Characteristics of included studies

We identified 23 relevant studies, of which 16 randomised controlled clinical trials8^–23 and one non-randomised controlled clinical trial24 met the inclusion criteria. All used reliable outcome measures with adequate blinding of assessment. No interrupted time series or studies controlled at a second site were identified. Sixteen studies used patients as the unit of allocation; one allocated medical firms to intervention and control.9 Three studies did not use the same unit for allocation and analysis. 9 12 16 Only two studies reported a power calculation. 11 15 Six studies reported adequate concealment of allocation (for example, random numbers in opaque envelopes),8^–13 12 followed up more than 80% of patients, 8 12 13 15^–20 22 24 25 12 reported similar baseline measures between intervention and control groups, 8^–14 16 1920 23 24 nine recorded that patient consent had been obtained, 8^–10 15 18^–20 22 23 and eight reported gaining approval from an ethics committee. 8 10 12 18^–20 22 24 In one study the reviewers thought that there was little room for improvement because the performance of the health professional was adequate without the intervention.23

Most studies were randomised by patient, so the same health professional might have given treatment to study and control groups. If the same person treated both groups it is possible that the effect of computer advice might have carried over into the control group. Such studies would tend to underestimate the effect of computer support. Two studies of continuous infusion anaesthesia, although randomised by patient, had a sufficiently rigorous study design to ensure that this contamination was unlikely to occur. 19 24 In these studies a computer controlled pump delivered the drug directly to the patient, and the anaesthetist administered additional anaesthetic agents without knowing the amount of drug given by the computer. In the study randomised by medical firm all firms worked at the same hospital, so the computer advice might have influenced treatment of the control group. The only studies judged to be free of contamination were the two studies of continuous infusion anaesthesia.

Types of computer support systems used

Most of the computer systems used a mathematical model of the pharmacokinetics of the drug in question to predict the required dose (table). These models represent the compartments in the body in which the drug is distributed. Rate constants are used to describe the movements of the drug between different compartments. The models ranged from a simple, one compartment model for theophylline14 to a more complex, three compartment model for fentanyl.24 The starting values for the rate constants were estimated from population data but could then be adjusted as data accumulated from an individual patient. The systems allowed the operator to specify a target blood concentration of drug, which the computer then attempted to achieve. When the effect of the drug was more important than its blood concentration, pharmacodynamic parameters based on population data were added to the model.22

Effects of computer support on outcome

The table shows the effects of computer support on the process and outcome of care.

Drug doses used —Eleven studies examined change in the drug dose when computer support was used, 8 9 11 13 14 17^–19 21 23 24 and seven found significant changes. 8 13 14 17 19 21 24Studies on theophylline showed increases in initial dose 14 21and in maintenance dose. 13 14 Studies on intravenous anaesthesia showed a reduction in total dose of fentanyl used24 and a reduction in initial dose and maintenance dose of midazolam.19

Drug concentrations within desired range —Of the seven studies that measured changes in drug concentrations in the body two found significant increases in the proportion of patients with drug concentrations in the therapeutic range with computer support. 9^–11 13 14 17 21

Physiological control —Eight studies measured changes in control of a physiological parameter with computer support, 8 15 16 18 20 22^–24 of which six showed significant benefit. 8 15 18 20 22 24 Computer support for anticoagulant control resulted in significant reductions in the time taken to achieve the desired prothrombin time22 and activated partial thromboplastin time.15 In postoperative control of blood pressure with sodium nitroprusside, a computer assisted pump was more effective at keeping blood pressure in the target range than a manually controlled infusion. Babies delivered to women treated with computer controlled oxytocin had a lower lactate concentration in the umbilical cord blood.8

Unwanted effects of drug treatment —Six studies measured the unwanted effects of drugs, 11 12 14 15 22 24 and four found significant reductions associated with computer support. 14 15 22 24 Fewer patients treated with theophylline reached toxic drug concentrations when computer advice was used.14 In studies on anticoagulation both the number of patients given too much anticoagulant22 and the total number of adverse events15 were reduced in the intervention groups. The number of hypotensive episodes during cardiac surgery was reduced when fentanyl and midazolam were given via a computer controlled pump.24

Cost of drug treatment —Only two studies reported economic data, and both looked at computer support for aminoglycoside dose. 9 11 In one study the mean direct cost of treatment with computer support was $7102 compared with $13 758 in controls (P<0.02), with a benefit to cost ratio of 75.11 The other calculated a cost avoidance (the money potentially saved by the intervention) of $1311 for each patient treated, with a benefit to cost ratio of 4.1.9 These cost savings resulted largely from reduced hospital stays, which was confirmed in one study,14 although another suggested an increased time spent in hospital.10 Another study showed that computer support lengthened the interval between outpatient visits.12

Outcome of medical care —Six studies directly measured outcomes of care, of which five showed benefits. Three showed significant benefits in clinical improvement scores for asthma,21 treating infection,9 and pain relief after surgery.20 Fewer caesarean sections were required when oxytocin was given by computer controlled pump to augment labour.9 With computer support for hospital treatment of acute asthma, fewer patients subsequently needed convalescent care,13 and there were fewer deaths.14 One study on anticoagulation showed an increase in the number of embolisms, but it may be that the computer system used in this study was set to achieve too low a prothrombin time.12

Overall effect —Eleven studies provided outcomes for quantitative synthesis, 10 11 13^–15 17^–19 21^–23 and the figure shows the individual results and meta-analysis for each of the four outcome measures. Patients treated with computer support had higher blood concentrations of the drug (effect size 0.69, 95% confidence interval 0.36 to 1.02) and took less time to reach therapeutic concentrations −(0.44,−0.71 to 0.17). However, computer support had no significant effect on the total amount of drug used (−0.43, −1.00 to 0.15) nor on the difference between the level of a physiological parameter achieved and the target level (−1.22, −3.31 to 0.87). Although the clinical settings of the trials varied widely, only in the case of difference from target level was there evidence of statistical heterogeneity.

Quantitative effects of computer support for determining drug dose on outcomes of care (standardised weighted mean differences (95% confidence intervals) for computer support versus control)

• Download figure
• Open in new tab
• Download powerpoint

Methodological issues

Potentially, the most important factor that may undermine our analysis is publication bias—studies with positive results are more likely to be published than those with negative results.26 Our search was exhaustive, and it seems unlikely that large numbers of patients have been randomised to trials that we have not identified. Orwin's file drawer method suggests that there would need to be 24 unpublished studies showing no benefit from computer support to alter our results significantly.27

Another review in a similar area did not use statistical methods to estimate the overall effects from computer support,28 preferring to present a “vote count” of statistically significant studies. This method risks concluding falsely that an important effect is absent because it assumes that a trial with no significant effect is a negative study.29 The method also does not make full use of available data. Our overview represents an advance from this approach, using established and robust statistical methods to estimate an overall treatment effect. 29 30

Our findings need to be read with caution since we identified relatively few studies on a limited range of drugs and our quantitative analysis was based on results derived from only 671 patients. Computer assisted determination of drug dose is a potentially hazardous intervention: it is surprising that the risks and benefits have not been evaluated in large randomised controlled trials. The quality of the studies could be improved: most did not report a power calculation, and sample sizes were often small. A common bias in many of the studies was that the same clinicians treated patients allocated to intervention and control, so the effect of the intervention would tend to spill over into the control group. Contamination of the control group in this manner would tend to make it more difficult to show a benefit from computer support.

More large scale studies are needed to confirm our conclusions, and research is needed to examine the effects of computer support in general use and to develop and implement appropriate systems. Existing studies suggest that computer support for drug dosage will be cost effective, but economic evaluation should be an integral part of any future study. Economic benefits seen in one therapeutic area may not transfer to another clinical situation.

Implications for computer support for drug dose

The scope of computer support systems could be widened to include other, commonly prescribed drugs with a narrow therapeutic window, such as anticonvulsants and lithium. A computer might use the same basic pharmacokinetic model for several different drugs. It is possible that the model could be extended to predict the likelihood and severity of some interactions—for example, those caused by competition for protein binding sites.

A barrier to adopting computer support may be the lack of access to suitable computers and electronic medical records. Computerised records are used in some hospital departments (such as intensive care), but hospitals often rely on paper to record outpatient treatment. In contrast, most general practitioners routinely use electronic records for prescribing31 and have easy access to suitable hardware that could run programs to help them to determine the most appropriate dose of commonly used drugs. General practice computers often already store necessary data such as blood concentrations of drugs, body mass index, and indicators of hepatic and renal function. It may be that the benefits seen in secondary care could be realised on a large scale if programs giving support for drug dose were integrated with the software routinely used in general practice.