Error in key model input

I offer the following comments regarding the recent modelling study published in CMAJ,1 which has several important shortcomings.

First, a model does not measure, observe or test anything. It is a prediction tool, and the authors themselves a described it as a “simple” model. The predictions of this model have not been tested so, at most, this model should be regarded as a hypothesis. Like any hypothesis, it needs to be tested and validated before its predictions should be considered evidence.

Second, the output of any model is totally dependent on the quality and accuracy of its inputs. The key input for this model is the vaccine effectiveness (VE) in preventing infection. The model assumes that this VE is 40%–80%.

The authors cite 2 references to support the lower bound (40%) estimate. The first is a surveillance report from the United Kingdom at a time (December 2021) when the Omicron variant was just emerging.2 The data in this report are based on the Delta variant, but the report makes it clear that lower VE with Omicron is anticipated. The second reference is simply another unvalidated model.3

The authors cite only a single reference to support an upper bound (80%) estimate of VE.4

The authors’ use of this single reference is highly problematic for 3 reasons. First, the reference covers data only up until Oct. 20, 2021 — 6 months ago! It does not take into account the impact of new variants (Omicron and B.2) or continuing waning immunity. Second, the reference study does not support a VE estimate of 80%. The reference study measured VE for 3 vaccines between July 1, 2021, and Oct. 20, 2021, at 49%, 52% and 70%.

Third, the authors have failed to acknowledge abundant new evidence,5^,6 including some from Ontario,7^,8 showing little or no persistent (and perhaps even negative) VE against infection.