Challenges to the evaluation of candidate vaccines for severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) before approval or licensure during the ongoing pandemic include rapidly changing levels of exposure to the virus and population immunity and social distancing practices.
To measure vaccine efficacy accurately, researchers should account for these factors in sample size calculations and carefully consider selection of trial end points.
Vaccines for SARS-CoV-2 must also be evaluated in populations known to be at increased risk for severe coronavirus disease 2019, such as older adults, Black people and people with multiple comorbidities.
Given the speed of vaccine development for SARS-CoV-2, careful attention must be paid to postlicensure assessment of vaccines, including the risk of antibody-dependent enhancement of disease, which must be actively monitored closely over multiple years after vaccination.
Coronavirus disease 2019 (COVID-19) is a rapidly evolving pandemic caused by severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). In addition to minimizing risk of transmission by nonpharmacologic measures, 1 or more effective vaccines would be invaluable to reduce the burden of COVID-19. More than 140 candidate SARS-CoV-2 vaccines are in development and being assessed in preclinical studies and clinical trials.1 In general, for a vaccine to be approved or licensed by regulatory authorities, it must demonstrate both safety and high efficacy in the prevention of a specific disease in the relevant populations. However, the COVID-19 pandemic poses specific logistic and scientific challenges with respect to the assessment of SARS-CoV-2 vaccine candidates. Evaluation of the efficacy of SARS-CoV-2 vaccines must consider population risk of exposure, susceptibility to the virus, current social distancing practices and geography. Moreover, SARS-CoV-2 vaccines need to be evaluated in populations at greatest risk for severe COVID-19.2–4 We discuss challenges to the clinical evaluation of SARS-CoV-2 vaccines, as well as some potential solutions.
How are vaccines usually evaluated before licensure?
Vaccines considered for licensure are assessed initially for their safety profile and their ability to induce immune responses to vaccine antigens in small phase 1 studies. Candidate vaccines that are safe and immunogenic may then advance to phase 2 studies, in which safety, immunogenicity and sometimes preliminary efficacy are assessed in larger cohorts. Vaccine efficacy is primarily evaluated in phase 3 studies to determine the proportionate reduction in predefined infection rate or disease events among vaccinated participants. Vaccine efficacy represents the best-case scenario of protection derived from vaccination and needs to be demonstrated for a vaccine against a novel pathogen to be licensed by regulatory authorities.2,3 Ideally, evaluation of efficacy is undertaken in well-controlled studies (e.g., double-blind, randomized, controlled clinical trials). Successful vaccines that have been shown to have high efficacy in clinical trials and are licensed for use in a population are continually assessed for their safety and effectiveness postlicensure and postimplementation (in phase 4 studies that evaluate how a vaccine reduces disease in a population in “real world” conditions).
What factors may affect the evaluation of candidate vaccines for SARS-CoV-2?
In general, host-related (e.g., age, genetic and environmental exposures) and vaccine-related (e.g., antigen selection, adjuvants, formulation, delivery mode and waning of immunity over time) factors influence individuals’ immune response to vaccines and thus determine the vaccines’ efficacy. However, several other factors, some of which may be difficult to quantify, such as the level of transmission of the target pathogen (i.e., exposure) and the level of pre-existing immunity (i.e., susceptibility) in the population are also important to consider. Such factors are related and are different when a pathogen is endemic (i.e., constantly and usually present within a given geographic area or population) versus pandemic (i.e., emerging and spreading worldwide), as with SARS-CoV-2. The baseline level of exposure of a population to a specific pathogen, the pathogen’s seasonality and the level of population immunity are usually relatively predictable in endemic compared with pandemic states of infection. Table 1 summarizes the potential confounders unique to the COVID-19 pandemic that might affect the demonstration of efficacy of candidate vaccines for SARS-CoV-2.
Population-related factors affecting evaluation of vaccine efficacy in endemic versus pandemic states of infection and those unique to coronavirus disease 2019
Level of exposure and immunity to SARS-CoV-2
The dynamic and rapidly changing pattern of virus exposure and level of population immunity during the evolving pandemic are potentially important confounders in the assessment of efficacy of SARS-CoV-2 vaccines; this should be considered in sample size calculations for efficacy trials. For example, if a vaccine is trialled in a low-incidence population, or if immunity wanes substantially over time postvaccination, a highly efficacious vaccine might not show significant protection if the trial sample is too small to demonstrate a significant reduction in occurrence of disease in vaccinated participants in such conditions. Thus, careful attention must be paid to trial sample size calculations, and there may be a need to recruit more participants in areas where the disease prevalence is very low. Moreover, levels of transmission of SARS-CoV-2 might vary between or within countries and will change over time as the pandemic progresses, which must be considered when trialling the same vaccine in different geographic areas.
Population seropositivity, or baseline level of immunity, might affect vaccine immunogenicity or population susceptibility to the target pathogen, which would influence the outcome of a vaccine trial. For example, a highly efficacious vaccine may not show benefit in settings with high seroprevalence, as vaccination may not add substantially to the protection afforded by natural infection. Although this may not be a problem very early in the pandemic while population seropositivity is still low,4 high baseline immunity could confound the results of vaccine trials as the pandemic progresses and seroprevalence increases, depending on the population. For example, the prevalence of anti-SARS-CoV-2 antibodies increased (from 3.1% to 6.1% to 9.7%) during 3 subsequent weeks in April 2020 in a population-based sample in Geneva, Switzerland.5 Measurement of pre-existing immunity in efficacy trials is important, as is the inclusion, vaccination and evaluation of seropositive participants.
Social distancing and other public health interventions
Social distancing has been shown to be effective in mitigating the transmission of SARS-CoV-2.6,7 In settings where transmission is low because of social distancing measures, the benefit of a highly efficacious vaccine might not be readily demonstrable. Study sample size calculations should account for this. Furthermore, public health interventions will likely vary as the pandemic progresses, which will challenge the ongoing assessment of vaccines. Flexibility in trial design may be necessary to achieve a sufficient number of end points to determine efficacy, depending on fluctuating transmission rates in different locations.
What potential vaccine-related harms may be anticipated?
Although the goal of a vaccine is to reduce the burden of COVID-19, SARS-CoV-2 vaccines may theoretically lead to antibody-dependent enhancement.8 This phenomenon, which has been described with SARS-CoV and other coronavirus vaccines in animal models,9 results from low-titre or poorly neutralizing antibody after vaccination that facilitates viral entry or replication in target cells, causing more severe disease from infection in vaccinated individuals. As shown with the CYD-TDV vaccine for dengue, vaccine-induced antibody-dependent enhancement can have enormous negative effects, not only directly among vaccinated individuals, but also on public trust and uptake of other vaccines.10,11 Antibody-dependent enhancement may be observed only after vaccination with specific SARS-CoV-2 vaccine adjuvants, platforms or products — not all. Importantly, vaccine-related antibody-dependent enhancement may become evident only when enough people have been vaccinated and there is high circulation of the virus to show a large burden of COVID-19 among the vaccine population, or it may accompany waning of a vaccine immune response in subsequent years or be related to genotypic changes in the virus over time.12 We suggest that the risk of antibody-dependent enhancement be actively monitored closely over multiple years to account for waning antibody titres or variation in circulating viral strains (Table 1).
Why is careful consideration of study population and trial end points important?
Vaccines for SARS-CoV-2, like other vaccines, are being first studied in populations of healthy adult volunteers. However, COVID-19 has been shown to affect older adults, Black people and people with multiple comorbidities most severely.13,14 Thus, results from low-risk populations may not reflect benefits or risks in higher-risk populations. It is also possible that a vaccine may not provide sterilizing immunity (defined as immune status after vaccination that prevents virus infection of the host). This means that vaccinated individuals may still get the mild form of the disease. Thus, it is possible that vaccination will have a substantially greater impact on preventing severe disease than on preventing milder symptoms or acquisition of infection. These issues should be considered when choosing the clinical end points for trials assessing SARS-CoV-2 vaccine efficacy. For example, if the aim is to show that a vaccine is effective in reducing the severe forms and outcomes of COVID-19, end points such as hospital admission, intensive care unit admission, the need for respiratory support and death could be considered as primary outcomes, although lower rates of more severe events, depending on overall disease incidence rates, may require more trial participants.
It is also important that vaccines be tested in the populations at greatest risk for severe COVID-19, such as older adults, health care workers, Black people and those with predisposing health problems.15 This is critical as it will inform policy-makers about the populations that will benefit the most from vaccination and thus need to be prioritized for vaccination when a vaccine is available but the demand outweighs the supply.
What are some other challenges to the development of SARS-CoV-2 vaccines?
Regulatory challenges
Development of vaccines for human use usually takes at least 10–15 years, from preclinical development to licensure. However, owing to the urgent need for SARS-CoV-2 vaccines, the timeline for their development and approval will need to be shortened substantially, ideally to months rather than years. This may impose a substantial burden on regulatory agencies to process all the usual work in a greatly reduced time frame.
Safety challenges
New technologies for vaccine delivery have been developed during the past decade (e.g., DNA or RNA vaccines); these technologies feature among the SARS-CoV-2 vaccines that have most rapidly progressed to clinical trials, despite their not having been used for any licensed vaccine to date.15 The safety of the SARS-CoV-2 candidate vaccines based on these new technologies must be assessed especially carefully. This will require continuous assessment for both common and rare potential adverse effects in a large number of vaccine recipients, during clinical trials as well as during active postmarketing surveillance.
What are some particular organizational challenges and opportunities of SARS-CoV-2 vaccine trials?
Logistics of multiple vaccines in trials
Studying multiple vaccines at the same time in the same population may be a challenge if there is limited trial capacity. This might be overcome by using an adaptive study design allowing addition or removal of vaccines as data on safety, immunogenicity and efficacy accumulate. This approach is currently being employed in clinical trials assessing different pharmacologic treatments for COVID-19 (e.g., the Solidarity Trial, launched by the World Health Association and partners16). Additionally, infrastructure is being developed to enable sites with experience of nonvaccine drug trials to be used, and for new trial sites to be rapidly set up, including appropriate focused training for research staff.
National and international collaborations
Establishing national and international consortia for SARS-CoV-2 vaccine trials will increase cooperation within and between countries’ vaccine groups. For example, the Canadian Immunization Research Network is a national collaborative network of vaccine researchers who conduct collaborative research related to different aspects of vaccinology. To ensure equity in evaluating vaccines, individual study sites should be representative of the entire population in a specific country, and evaluation of vaccines should be performed in high- as well as low- and middle-income countries. As demand for efficacious vaccines will be high, setting parameters for equitable postlicensure distribution of vaccines across countries is also critical.
How should postlicensure assessment of vaccines be conducted?
Postlicensure assessment of the effectiveness of a vaccine that is being administered in the population is mainly done by retrospective case–control analysis, or ecologic or observational studies. Analyses of these studies should include populations with varying risk of exposure, pre-existing immunity (if known), geography and baseline characteristics (e.g., age and comorbidities). Time since vaccination can also be captured in these studies to assess the effectiveness of vaccination with SARS-CoV-2 vaccines over time after vaccination. This will help to determine whether there is waning immunity after vaccination.
Conclusion
We have discussed the unique challenges in evaluating SARS-CoV-2 vaccines during the ongoing pandemic. These arise from rapidly changing levels of virus exposure, the development of population immunity and the effects of local social distancing practices, as well as the speed with which vaccine candidates are being developed and tested, safety considerations of untested technologies, and the need to consider which end points matter, who is most affected and issues of equity in access to approved vaccines. In addition, as efficacy might be challenging to demonstrate in some populations, researchers should aim to fully dissect the protective immune response (humoral and cellular immunity) to natural infection and after vaccination, as establishing correlates of protection would also inform vaccine development and evaluation.
Acknowledgements
Bahaa Abu-Raya is supported by the Canadian Health and Research Institute Vanier Canada Graduate scholarship. Manish Sadarangani is supported via salary awards from the BC Children’s Hospital Foundation, the Canadian Child Health Clinician Scientist Program and the Michael Smith Foundation for Health Research.
Footnotes
Competing interests: Soren Gantt receives consulting fees related to vaccines from Moderna, Merck, GlaxoSmithKline and Curevo; research funding for vaccine research from Merck, GlaxoSmithKline and VBI; and research funding for a COVID-19 vaccine trial from Symvivo. Soren Gantt is also a voting member of the Canadian National Advisory Committee on Immunization. Manish Sadarangani has been an investigator on projects funded by Pfizer, Merck, Seqirus, Sanofi Pasteur, VBI Vaccines and GlaxoSmithKline, and has received research funding for a COVID-19 vaccine trial from Symvivo. All funds have been paid to his institution, and he has not received any personal payments. No other competing interests were declared.
This article has been peer reviewed.
Contributors: All authors contributed to the conception and design of the work. Bahaa Abu-Raya drafted the manuscript. All authors critically reviewed and revised the article, gave final approval of the final version to be published and agreed to be accountable for all aspects of the work.
References
In this issue
Article tools
Jump to section
- Article
- How are vaccines usually evaluated before licensure?
- What factors may affect the evaluation of candidate vaccines for SARS-CoV-2?
- What potential vaccine-related harms may be anticipated?
- Why is careful consideration of study population and trial end points important?
- What are some other challenges to the development of SARS-CoV-2 vaccines?
- What are some particular organizational challenges and opportunities of SARS-CoV-2 vaccine trials?
- How should postlicensure assessment of vaccines be conducted?
- Conclusion
- Acknowledgements
- Footnotes
- References
- Figures & Tables
- Related Content
- Responses
- Metrics
Related Articles
Cited By...
- No citing articles found.