Grading of Recommendations, Assessment, Development, and Evaluation (GRADE): Novavax COVID-19 Vaccine

Overview

A Grading of Recommendations, Assessment, Development and Evaluation (GRADE) review of the evidence for benefits and harms for Novavax coronavirus disease 2019 (COVID-19) vaccine was presented to the Advisory Committee on Immunization Practices (ACIP) on July 19, 2022. GRADE evidence type indicates the certainty in estimates from the available body of evidence. Evidence certainty ranges from type 1 (high certainty) to type 4 (very low certainty)1.

The policy question was, "Should vaccination with Novavax COVID-19 vaccine be recommended for persons 18 years of age and older during an Emergency Use Authorization?" The potential benefits pre-specified by the ACIP COVID-19 Vaccines Work Group were prevention of symptomatic laboratory-confirmed COVID-19 (critical), hospitalization due to COVID-19 (critical), death due to COVID-19 (important), and asymptomatic SARS-CoV-2 infection (important). The two pre-specified harms were serious adverse events (critical) and reactogenicity grade ≥3 (important).

A systematic review of evidence on the efficacy and safety of a two-dose regimen of Novavax COVID-19 vaccine among persons aged 18 years and older was conducted. The quality of evidence from one Phase III randomized controlled trial was assessed using a modified GRADE approach.23

A lower risk of symptomatic COVID-19 was observed with vaccination compared to placebo (relative risk [RR] 0.10, 95% confidence interval [CI]: 0.06, 0.18, evidence type 1), corresponding to a vaccine efficacy of 89.6% (95% CI: 82.4%, 93.8%). This was observed with a median follow-up of 2.5 months, during a period of Alpha variant predominance. The vaccine was also associated with a lower risk of severe illness due to COVID-19 (RR 0.00; 95% CI: 0.00, 1.00; evidence type 3), corresponding to a vaccine efficacy of 100% (95% CI: 0%, 100%). The measure of severe COVID-19 was used as surrogate for the GRADE outcome of hospitalization due to COVID-19. No hospitalizations or deaths due to COVID-19 were identified among vaccine recipients or placebo recipients in the per-protocol population.*

In terms of harms, the available data indicated that serious adverse events were balanced between the vaccine and placebo arms (RR 0.92; 95% CI: 0.73, 1.16; evidence type 1). Reactogenicity grade ≥3 was associated with vaccination (RR 4.11; 95% CI: 3.70, 4.57; evidence type 1), 16.3% of vaccine recipients and 4% of placebo recipients reported any grade ≥3 local or systemic reactions following either dose 1 or dose 2.

Introduction

On July 13, 2022, the U.S. Food and Drug Administration (FDA) issued an Emergency Use Authorization (EUA) for Novavax COVID-19 (NVX-CoV2373) vaccine for prevention of symptomatic COVID-19 for persons aged ≥18 years.4 As part of the process employed by the ACIP, a systematic review and GRADE evaluation of the evidence for Novavax COVID-19 vaccine was conducted and presented to ACIP. The ACIP adopted a modified GRADE approach in 2010 as the framework for evaluating the scientific evidence that informs recommendations for vaccine use. Evidence of benefits and harms were reviewed based on the GRADE approach.1

The policy question was, "Should vaccination with Novavax COVID-19 vaccine be recommended for persons 18 years of age and older under an Emergency Use Authorization?" (Table 1).

Methods

We conducted a systematic review of evidence on the efficacy and safety of a two-dose regimen of Novavax (5 μg antigen plus 50 μg Matrix-M adjuvant) COVID-19 vaccine. We assessed outcomes and evaluated the quality of evidence using the GRADE approach.

During Work Group calls, members were asked to pre-specify and rate the importance of relevant patient-important outcomes (including benefits and harms) before the GRADE assessment. No conflicts of interest were reported by CDC and ACIP COVID-19 Vaccines Work Group members involved in the GRADE analysis. Outcomes of interest included individual benefits and harms (Table 2). The critical benefits of interest are prevention of symptomatic laboratory-confirmed COVID-19 and prevention of hospitalization due to COVID-19. Other important outcomes include prevention of death due to COVID-19 and prevention of asymptomatic SARS-CoV-2 infection. The critical harm of interest was serious adverse events; reactogenicity grade ≥3 was deemed an important harm.

We identified clinical trials through clinicaltrials.gov. Records of relevant Phase I, II, or III RCTs of COVID-19 vaccine were included if they 1) provided data on persons aged ≥18 years vaccinated with NVX-CoV2373; 2) involved human subjects; 3) reported primary data; and 4) included data relevant to the efficacy and safety outcomes being measured. We identified relevant observational studies through an ongoing systematic review conducted by the International Vaccine Access Center (IVAC) and the World Health Organization (WHO).4 Relevant observational studies were restricted to the defined population, intervention, comparison, and outcome outlined in the policy question, or related outcomes if direct data were not available. In addition, unpublished and other relevant data were obtained by hand-searching reference lists, and consulting with vaccine manufacturers and subject matter experts. The systematic review was limited to studies published from January 1, 2020 to July 7, 2022. Characteristics of all included studies are shown in Appendix 1 and evidence retrieval methods are found in Appendix 2.

The evidence certainty assessment addressed risk of bias, inconsistency, indirectness, imprecision, and other characteristics. The GRADE assessment across the body of evidence for each outcome was presented in an evidence profile; the evidence certainty of Type 1, 2, 3, or 4 corresponds to high, moderate, low, or very low certainty, respectively.

Relative risks (RR) were calculated from numerators and denominators available in the body of evidence. Vaccine efficacy estimates were defined as 100% x (1-RR).

GRADE Summary

The initial GRADE evidence level was type 1 (high) for each outcome because the body of evidence was from a randomized controlled trial (Table 4). In terms of critical benefits, the available data indicated that the vaccine was effective for preventing symptomatic COVID-19 during a period of Alpha variant predominance, and no serious concerns impacting certainty in the estimate were identified (type 1, high). The certainty in the effect estimate for severe illness due to COVID-19 was downgraded one point for serious concern of indirectness because severe illness due to COVID-19 was evaluated as a surrogate for hospitalization due to COVID-19 and one point for serious concern of imprecision due to the small number of events during the observation period (type 3, low). No serious concerns impacted the certainty in the estimate for serious adverse events or reactogenicity (both type 1, high) (Table 4).

* Cases were included in the analysis if they were confirmed in a designated central laboratory, per manufacturer protocol

^† Additional studies which evaluated the Novavax vaccine were excluded as the vaccines were manufactured at a different facility and by a different process

Tables

Table 1: Policy Question and PICO

Abbreviations: IM = intramuscular.

^aAssessed through serial PCR testing.

Table 2: Outcomes and Rankings

^aSevere illness due to COVID-19 evaluated as a surrogate measure for this critical outcome.

^bNo events occurred in the study included in the review of evidence.

^cData were not available to inform an evaluation of asymptomatic SARS-CoV-2 infection in studies identified in the review of evidence.

Table 3a: Summary of Studies Reporting Symptomatic Laboratory-confirmed COVID-19

References in this table: 23

Abbreviations: RT-PCR = real-time polymerase chain reaction; CI = confidence interval; RR = relative risk.

^a19,965 and 9,984 persons were randomized to vaccine and placebo

^bPrimary outcome, defined as SARS-CoV-2 RT-PCR-positive symptomatic illness, in seronegative adults, ≥7 days post vaccination.

^cSymptomatic illness defined as any mild, moderate or severe COVID-19. Mild COVID-19 was defined as fever, new onset cough OR ≥2 additional COVID-19 symptoms: pyretic, new onset or worsening of shortness of breath or difficulty breathing compared to baseline, new onset fatigue, new onset generalized muscle or body aches, new onset headache, new loss of taste or smell, acute onset of sore throat, congestion, and runny nose, or new onset nausea, vomiting, or diarrhea. Moderate COVID-19 was defined as high fever (≥38.4°C) for ≥3 days or any evidence of significant lower respiratory tract infection. Severe COVID-19 was defined as any of the following symptoms: tachypnea: ≥ 30 breaths per minute at rest, resting heart rate ≥125 beats per minute, oxygen saturation ≤93% on room air or ratio of the partial pressure of arterial oxygen to the fraction of inspired oxygen <300 mm Hg, high flow oxygen therapy or non-invasive ventilation/non-invasive positive pressure ventilation, mechanical ventilation or extracorporeal membrane oxygenation, one or more major organ system dysfunction or failure.

^dBased on data cutoff September 27, 2021; participants had a median of two months follow-up.

Table 3b: Summary of Studies Reporting Severe Illness due to COVID-19^a,b

References in this table: 3

Abbreviations: CI = confidence interval; RR = relative risk; NE=Not estimable

^a Severe COVID-19 was defined as any of the following symptoms: tachypnea: ≥ 30 breaths per minute at rest, resting heart rate ≥125 beats per minute, oxygen saturation ≤93% on room air or ratio of the partial pressure of arterial oxygen to the fraction of inspired oxygen <300 mm Hg, high flow oxygen therapy or non-invasive ventilation/non-invasive positive pressure ventilation, mechanical ventilation or extracorporeal membrane oxygenation, one or more major organ system dysfunction or failure.

^bBased on data cutoff September 27, 2021.

Table 3c: Summary of Studies Reporting Serious Adverse Events^a,b

References in this table: 3

Abbreviations: RR = relative risk; CI = confidence interval; RCT = randomized controlled trial.

^aDeath, life-threatening event, hospitalization, incapacity to perform normal life functions, medically important event, or congenital anomaly/birth defect

^bFive participants in the vaccine arm experienced SAEs that were considered related to vaccination by the investigator (n=1 each of headache, angioedema, Basedow's disease, thrombocytopenia, nervous system disorder). Among these, FDA considered the event of angioedema as potentially related to vaccination. There was one event of myocarditis in a 67-year-old male, with concomitant COVID-19 infection, 28 days after dose 1, which was not considered related to vaccination. Additional cases of myocarditis were observed after placebo crossover

Table 3d: Summary of Studies Reporting Reactogenicity^a,b

References in this table: 3

Abbreviations: RR = relative risk; CI = confidence interval.

^aReactogenicity outcome includes local and systemic events, grade ≥3. For local reactions, grade 3 pain or tenderness defined as any narcotic pain reliever or prevents daily activity, grade 4 defined as emergency department visit or hospitalization. Grade 3 redness or swelling is defined as >10 cm or prevents daily activity, grade 4 is necrosis or exfoliative dermatitis. For systemic reactions, grade 3 fever defined as 39.0 to 40.0 °C, grade 4 defined as >40°C. Grade 3 headache, fatigue/malaise, muscle pain, and joint pain defined as any use of narcotic pain reliever or prevents daily activity, grade 4 defined as emergency department visit or hospitalization. Grade 3 nausea/vomiting defined as prevents daily activity or requires outpatient IV hydration, grade 4 defined as emergency department visit or hospitalization for hypotensive shock.

^bBased on interim analysis, data cutoff September 27, 2021.

Table 4. Grade Summary of Findings Table

Abbreviations: CI = confidence interval; RR = relative risk; COVID-19 = coronavirus disease 2019; RCT = randomized controlled trial.

a. Risk of bias related to blinding of participants and personnel was present. Although participants and study staff were blinded to intervention assignments, they may have inferred receipt of vaccine or placebo based on reactogenicity. This was deemed unlikely to overestimate efficacy or underestimate risk of serious adverse events, therefore the risk of bias was rated as not serious.

b. Concern for indirectness was noted due to the short duration of observation in the available body of evidence. The vaccine efficacy observed at a median 2-month follow-up may differ from the efficacy observed with ongoing follow-up. However, in consideration that this recommendation is under an Emergency Use Authorization, the length of follow up time was deemed sufficient to support efficacy in that context. Additionally, it should be noted that the efficacy assessment took place during Alpha variant predominance and efficacy may differ for other variants.

c. The effects noted are from a per protocol analysis with outcomes assessed at least 7 days post dose 2 among persons who received two doses and had no evidence of prior SARS-CoV-2 infection.

d. The RCT excluded persons with prior COVID-19 diagnosis, pregnant women or women planning to become pregnant within 3 months of vaccination, and persons who were immunocompromised due to conditions and/or treatments (participants with stable/well-controlled HIV infection were not excluded). The population included in the RCT may not represent all persons aged ≥18 years.

e. Absolute risk was calculated using the observed outcomes in the placebo arm during the available clinical trial follow-up. Absolute risk estimates should be interpreted in this context.

f. Serious concern for indirectness was noted because severe illness due to COVID-19 is being evaluated as a surrogate for hospitalization due to COVID-19.

g. Serious concern for imprecision was noted due to the small number of events during the observation period.

h. There were no events in the vaccine group, therefore the relative risk was calculated using the standard offset of 0.5.

i. Absolute risk based on relative risk calculated using an offset due to zero events in the vaccine group

Appendix 1: Studies Included in the Review of Evidence

References in this table: 23

^aAdditional data provided by sponsor.References in this table:

Abbreviations: SD = standard deviation; RCT = randomized controlled trial; COVID-19 = coronavirus disease 2019.

Appendix 2. Databases and strategies used for systematic review

a. Most recent search conducted July 11, 2022.