Flawed analysis, implausible results — move on

Test 1: Common sense and empirical evidence

Hill finds that the annualized death rate while waiting in line for coronary surgery is the same whether priority is given to high- or low-risk patients, or neither. But common sense suggests that prioritizing low-risk patients would be dangerous, given known variations in subgroup-specific risks of cardiac death.3 Health care queuing systems worldwide prioritize patients with more symptoms or higher risks of irreversible events so as to reduce suffering and death. Even in a natural disaster or war, where some patients with a minimal probability of survival are triaged and left to die, priority among the remaining victims is given to those with more severe injuries.

Providers in the Cardiac Care Network of Ontario assess relative priorities and target waiting times with a patient-specific risk algorithm.4 Among 22 655 consecutive patients in Ontario registered for isolated coronary artery bypass grafting, independent risk factors for death while waiting included impaired left ventricular function, advancing age, male sex and, tellingly, waiting longer than the maximum time recommended for the relevant patient profile.5 Hill's analysis flies in the face of this empirical experience.

Test 2: Widgets versus patients

Hill also claims that prioritizing low-risk rather than high-risk patients would reduce mean waiting times from 365 days to 61 days, without any increase in throughput. How can this result be achieved? Think of 2 assembly lines in a factory, feeding 1 conveyor belt. One line makes widgets twice as fast as the other makes bolts. To avoid having widgets pile up, the supervisor has 2 workers loading widgets onto the conveyor and 1 worker loading bolts. There in a nutshell is Hill's model. Because the system receives more low-risk than high-risk patients, the former are given priority. If only queuing dynamics in health care were so straightforward.

Instead of widgets and bolts, think of tomatoes and eggs at different risks of bruising, breakage and rot, arriving in numbers that vary sharply from one day to the next. No supervisor would ever make a fixed assignment of workers to each assembly line and hope to keep her job!

But matters are even more complicated with coronary patients. Their risk of irreversible events (sudden cardiac death or myocardial infarction) is individualized, as is their time-dependent burden from symptoms. Although the patients can be aggregated into subgroups that arrive in more or less similar proportions on average from one year to the next, there are major week-to-week variations in the rates of arrival of different types of patients. Worse yet, patients switch categories: “tomatoes” become “eggs” when they suffer unstable angina in the queue.

In these circumstances, it is much more effective and efficient to devise a system focused on risk assessment and individualized maximum waiting times. And that is exactly what the Cardiac Care Network of Ontario does. It was designed from the outset with all these dynamic parameters and principles in mind.

Hill's elegant model, in contrast, is tantamount to a pair of misleading snapshots, one at the start of the year and another at the end, when in fact the behaviour of a queue can only be understood as a motion picture recorded over all 365 days. The model uses algebraic tautologies and static assumptions to achieve its seductive result.

Test 3: Internal logic, simulations and queuing theory

Hill's model makes total deaths in line (D) identical for each queuing paradigm by correlating D with T (waiting times) and Q (number of people in the queue). By algebraic substitution, Hill also concludes that Q = (N – S)/m, where m reflects the annualized mortality rates for the patient subgroups. Thus, patient-specific mortality risk supposedly determines the waiting list. Of course, m is biologically predetermined and has no direct impact on Q, except perhaps insofar as prioritizing patients with lower risk may perversely reduce Q by killing high-risk patients and increasing D, the total deaths in line. But if that is how Hill's elective-first scheme would shorten the queue, the total deaths in line over time would have to be higher — and Hill claims they would not be.

One author of this commentary (R.T.) wrote an S-PLUS computer program (S-PLUS 6.2, Insightful Corporation, Seattle, Wash.) to simulate survival among thousands of patients arriving for coronary surgery over a 3-year period, handled as Hill posits. There were very minor increases in mortality among low-risk patients when high-risk patients were given priority. However, there were major decrements in the probability of survival for high-risk patients with random allocation and especially with prioritization of low-risk patients. The result was that total survival probabilities fell dramatically with upside-down prioritization. In other words, once one sets aside Hill's 1-year snapshots and models the queuing paradigms empirically, a different result emerges.

Another author (M.S.) assessed Hill's model in the light of queuing theory. The situation considered by Hill is technically termed a non-pre-emptive, multiple-server priority queue with reneging.6 Hill's model is grossly deficient from an industrial engineering standpoint in that it completely ignores potential variability in any of its parameters. The model assumes wrongly that the queue is independent of its history, and does not appropriately account for the time patients have already spent in line or the effect of history on death rates. Even by its own restrictive and steady-state assumptions, the model miscalculates D. If D is to be calculated from Q, we need the value of Q at the beginning of the period (i.e., the original steady-state values of the queue). Instead, D is calculated in order to mitigate history (i.e., D is set at 40, while N = 1000 and S = 960). But the model tells us that D is dependent on Q, not the other way around. Moreover, none of the scenarios in Hill's Table 1 represent steady states. Simply projecting forward the values used by Hill for this third scenario, the low-risk priority approach, the number of deaths rises and the queue length changes the following year.

Conclusions

Hill presents his paper as a thought experiment, designed primarily to challenge conventional wisdom. By our reckoning, his upside-down queuing scheme would kill countless patients nationally and internationally, with sporadic and limited impacts on waiting times. But Hill's analysis does remind us that, if applied by experts in a safe and sensible fashion, queuing theory and logistics could have many benefits for the organization of health care. Those benefits can only be realized by more collaboration among clinicians, managers, epidemiologists/biostatisticians, logistics specialists and industrial engineers. These transdisciplinary collaborations are also good fun, and for catalyzing one such collaboration in this commentary, we thank Hill and the editors.

𝛃 See related article page 354