Free access
Original Research
20 July 2021

SARS-CoV-2 Vaccine Effectiveness in a High-Risk National Population in a Real-World SettingFREE

Publication: Annals of Internal Medicine
Volume 174, Number 10
Visual Abstract. SARS-CoV-2 Vaccine Effectiveness in a High-Risk National Population.
Data are urgently needed on the real-world effectiveness of the currently administered COVID-19 vaccines. This case–control study evaluated the short-term effectiveness of currently administered vaccines in preventing confirmed SARS-CoV-2 infection among veterans receiving care.

Abstract

Background:

With the emergency use authorization of multiple vaccines against SARS-CoV-2 infection, data are urgently needed to determine their effectiveness in a real-world setting.

Objective:

To evaluate the short-term effectiveness of vaccines in preventing SARS-CoV-2 infection.

Design:

Test-negative case–control study using conditional logistic regression.

Setting:

U.S. Department of Veterans Affairs health care system.

Participants:

All veterans who had testing for SARS-CoV-2 infection between 15 December 2020 and 4 March 2021 and no confirmed infection before 15 December 2020.

Intervention:

SARS-CoV-2 vaccination with either the BNT-162b2 (Pfizer–BioNTech) or mRNA-1273 (Moderna) vaccine as part of routine clinical care.

Measurements:

Effectiveness of vaccination against confirmed SARS-CoV-2 infection.

Results:

Among 54 360 persons who tested positive and 54 360 propensity score–matched control participants, the median age was 61 years, 83.6% were male, and 62% were White. Median body mass index was 31 kg/m2 among those who tested positive and 30 kg/m2 among those who tested negative. Among those who tested positive, 9800 (18.0%) had been vaccinated; among those who tested negative, 17 825 (32.8%) had been vaccinated. Overall vaccine effectiveness 7 or more days after the second dose was 97.1% (95% CI, 96.6% to 97.5%). Effectiveness was 96.2% (CI, 95.5% to 96.9%) for the Pfizer–BioNTech BNT-162b2 vaccine and 98.2% (CI, 97.5% to 98.6%) for the Moderna mRNA-1273 vaccine. Effectiveness remained above 95% regardless of age group, sex, race, or presence of comorbidities.

Limitations:

Predominantly male population; lack of data on disease severity, mortality, and effectiveness by SARS-CoV-2 variants of concern; and short-term follow-up.

Conclusion:

Currently used vaccines against SARS-CoV-2 infection are highly effective in preventing confirmed infection in a high-risk population in a real-world setting.

Primary Funding Source:

None.
In randomized, placebo-controlled clinical trials, vaccines developed by Pfizer–BioNTech (BNT-162b2) and Moderna (mRNA-1273) have shown 94% to 95% efficacy in preventing symptomatic SARS-CoV-2 infection (1, 2). The effectiveness against infection is more difficult to estimate and was lower in the earlier phase 3 clinical trials. A recent report from Israel showed 92% effectiveness of the BNT-162b2 vaccine in preventing documented infection and severe disease 7 or more days after the second dose (3). In another recent report from Qatar, the BNT-162b2 vaccine was 89.5% effective in preventing infection with the Alpha (previously known as the B1.1.7) variant and 75% effective in preventing infection with the Beta (previously known as the B1.351) strain (4). Other reports from select population groups have noted a sharp decrease in infections among vaccinated persons, although vaccine effectiveness is often not explicitly stated (5–7). Real-world data from large, national populations are urgently needed to provide reassurance about the effectiveness of current vaccines.
The veterans enrolled in care in the U.S. Department of Veterans Affairs (VA) health care system are at higher risk for severe clinical consequences due to older age, higher burden of comorbidities, and higher prevalence of social vulnerability factors (8–10). Therefore, the VA health care system provides a unique real-world setting to study natural history, disease outcomes, and effectiveness of new therapies in a high-risk population (11). We sought to determine the short-term effectiveness of currently administered vaccines in preventing confirmed SARS-CoV-2 infection among veterans receiving care in the VA health care system.

Methods

VA COVID-19 Shared Data Resource

In response to the SARS-CoV-2 pandemic, the VA rapidly created a national VA COVID-19 Shared Data Resource. Using case definitions and data mapping, which were validated collaboratively across the VA, the resource contains information on all veterans with a confirmed laboratory diagnosis of SARS-CoV-2 infection within the VA and those who were tested outside the VA with a VA clinical note confirming the diagnosis. The VA COVID-19 Shared Data Resource is updated regularly and contains extensive demographic, clinical, pharmacologic, laboratory, vital sign, and clinical outcome information derived from multiple validated sources, including the VA Corporate Data Warehouse and the VA electronic medical record.

Study Population and Data Set Creation

For the current study, we identified all veterans in the VA COVID-19 Shared Data Resource who had SARS-CoV-2 testing between 15 December 2020 and 4 March 2021 and no confirmed infection before 15 December 2020. For each person who tested positive, we identified a propensity score–matched control participant who tested negative, matched by age, sex, race, body mass index, Charlson Comorbidity Index score, and geographic location. We used nearest-neighbor matching with a caliper of 0.25 SD. The Charlson Comorbidity Index is a validated and widely used score that identifies persons at the highest risk for mortality over a period of time based on the presence and severity of comorbidities (12–14). For geographic location, we used the VA facility where the index test was performed to account for geographic variation in disease incidence and testing patterns. Data retrieved included demographic characteristics, clinical diagnoses, presence of symptoms at presentation, anthropometric measurements and vital signs, all pharmacologic interventions, and select laboratory results. Data on vaccine administration date and the type of vaccine used were also retrieved. Comorbidities were defined according to the definitions used in the VA Corporate Data Warehouse, which uses the International Classification of Diseases (Ninth or 10th Revision, as appropriate) for classifying comorbidities. Baseline diagnoses included all diagnoses recorded in the 2 years before the index date.

Statistical Analysis

We used a test-negative case–control design to evaluate the effectiveness of vaccination against confirmed SARS-CoV-2 infection. This is a widely accepted design for determining vaccine effectiveness in a population after the introduction of a vaccine (15–17). We used conditional logistic regression to calculate the odds of testing positive among the vaccinated group versus the unvaccinated group (18, 19). Vaccine effectiveness was calculated by using the following formula:
1 − Odds(T+|Vaccinated) / Odds(T+|Nonvaccinated)
Our primary outcome of interest was overall vaccine effectiveness 7 or more days after the second dose. We also assessed vaccine effectiveness among those who received only 1 dose and effectiveness anytime after receipt of the second dose. We evaluated effectiveness separately among those who received the Pfizer–BioNTech BNT-162b2 vaccine and those who received the Moderna mRNA-1273 vaccine.
Finally, we performed subgroup analyses by age group, sex, race, and comorbidity burden to determine vaccine effectiveness in various population subgroups. Effectiveness was calculated for each subgroup, and P values were calculated to test the significance of the difference within each subgroup. A P value less than 0.05 was considered significant. For all point estimates of vaccine effectiveness, we calculated 95% CIs (19, 20).

Ethical Considerations

The study was approved by the Institutional Review Board at the VA Pittsburgh Healthcare System. A waiver of informed consent was granted for the study.

Role of the Funding Source

There was no external funding source for this study.

Results

We identified 54 360 matched pairs of veterans who were tested for SARS-CoV-2 infection between 15 December 2020 and 4 March 2021 (1:1 ratio of positive and negative results). Baseline characteristics of the study population are shown in Table 1. The median age was 62 years among those who tested negative and 61 years among those who tested positive; in both groups, 83.6% of participants were male and 62% were White. Median body mass index was 31 kg/m2 among those who tested positive and 30 kg/m2 among those who tested negative. Among those who tested positive, 9800 (18.0%) had been vaccinated, and among those who tested negative, 17 825 (32.8%) had been vaccinated. The overall vaccine effectiveness was 97.1% (95% CI, 96.6% to 97.5%) 7 or more days after the second dose. Effectiveness was 96.2% (CI, 95.5% to 96.9%) for the Pfizer–BioNTech BNT-162b2 vaccine and 98.2% (CI, 97.5% to 98.6%) for the Moderna mRNA-1273 vaccine. The median follow-up after the first dose was 60 days (interquartile range [IQR], 49 to 69 days) for recipients of the Moderna mRNA-1273 vaccine and 56 days (IQR, 47 to 64 days) for recipients of the Pfizer–BioNTech BNT-162b2 vaccine. Among those who received 2 doses, median follow-up after the second dose was 30 days (IQR, 21 to 41 days) for recipients of the Moderna mRNA-1273 vaccine and 34 days (IQR, 26 to 43 days) for recipients of the Pfizer–BioNTech BNT-162b2 vaccine.
Table 1 Baseline Characteristics of Persons Who Tested Positive and Propensity Score–Matched Control Participants Who Tested Negative
Among those who received only 1 dose, effectiveness was 85.0% (CI, 84.2% to 85.8%) overall, 84.0% (CI, 82.7% to 85.1%) for the Pfizer–BioNTech BNT-162b2 vaccine, and 85.7% (CI, 84.6% to 86.8%) for the Moderna mRNA-1273 vaccine. Effectiveness against infection anytime after the second dose was 95.3% (CI, 94.7% to 95.9%) overall, 94.3% (CI, 93.4% to 95.2%) for the Pfizer–BioNTech BNT-162b2 vaccine, and 96.6% (CI, 95.7% to 97.2%) for the Moderna mRNA-1273 vaccine (Table 2).
Table 2 Vaccine Effectiveness, Calculated Using the Test-Negative Case–Control Design (Conditional Logistic Regression Analysis)
Vaccine effectiveness was numerically similar (though statistically significantly higher) among persons aged 70 years or older compared with those younger than 70 years (97.5% and 96.4%, respectively; P = 0.032). Effectiveness was similar between Blacks and Whites (97.4% and 96.9%, respectively; P = 0.20 overall for race) and between men and women (97.2% and 95.4%, respectively; P = 0.25). Vaccine effectiveness was 98.1% among persons with a Charlson Comorbidity Index score less than 2 and 96.9% among those with a score of 2 or higher (P = 0.025) (Table 3).
Table 3 Vaccine Effectiveness, by Population Subgroup

Discussion

In a large national health care system, we found the currently used SARS-CoV-2 vaccines to be more than 95% effective in preventing confirmed infection. Veterans are at particularly high risk given their older age and greater burden of comorbidities compared with the general population (8, 9), and the high level of vaccine effectiveness we observed is therefore reassuring. Our results are similar to those of a recent study at a national level in Israel, which found effectiveness of 92% for infection prevention with the Pfizer–BioNTech BNT-162b2 vaccine (3). Although these 2 studies had different designs and assessed different populations, the remarkably similar results provide further reassurance of the effectiveness of the vaccines in preventing infection in various real-world settings.
Vaccine effectiveness was 97.5% in persons aged 70 years or older and 96.4% among those younger than 70 years. Although this difference was statistically significant, the numerical similarity implies similarly high rates of effectiveness. Slightly higher effectiveness in older persons may be related to lower mobility and consequently a lower likelihood of exposure to infection. Older age has been associated with reduced mobility, which is in turn associated with increases in health care use, nursing home admissions, and mortality (21–23). There were no differences based on sex or race, with very high effectiveness among all subgroups. However, effectiveness was slightly lower among persons with comorbidities compared with those without comorbidities. Previous studies have shown a higher risk for infection as well as poorer clinical outcomes among persons with comorbidities. Our results are consistent with prior studies and the current understanding of SARS-CoV-2 infection.
Although we clearly show the effectiveness of the current vaccines in preventing infection, we did not assess their effectiveness in preventing severe disease and death. A cohort design is better suited to answer this question, as shown by the aforementioned study in Israel (3). Further studies are warranted to confirm this, although limited data from other health care systems suggest a high level of effectiveness in preventing these serious outcomes (3).
Strengths of our study include the large national study population at high risk for infection and use of the widely accepted test-negative case–control design. However, several limitations must be acknowledged, including a predominantly male population and a lack of data on disease severity and mortality. We did not have information on the reasons for testing, and we were unable to test the effect of SARS-CoV-2 variants of concern on vaccine effectiveness. Finally, we determined vaccine effectiveness over a relatively short period, and longer follow-up may be needed to determine long-term effectiveness.
In conclusion, the SARS-CoV-2 vaccines currently used in the VA health care system provided a high level of protection against confirmed infection. Further studies are needed to determine vaccine effectiveness in other populations (such as women and younger persons) and the effect of vaccination on other serious outcomes.

References

1.
Baden LREl Sahly HMEssink Bet alCOVE Study Group. Efficacy and safety of the mRNA-1273 SARS-CoV-2 vaccine. N Engl J Med. 2021;384:403-16. [PMID: 33378609]  doi: 10.1056/NEJMoa2035389
2.
Polack FPThomas SJKitchin Net alC4591001 Clinical Trial Group. Safety and efficacy of the BNT162b2 mRNA Covid-19 vaccine. N Engl J Med. 2020;383:2603-15. [PMID: 33301246]  doi: 10.1056/NEJMoa2034577
3.
Dagan NBarda NKepten Eet al. BNT162b2 mRNA Covid-19 vaccine in a nationwide mass vaccination setting. N Engl J Med. 2021;384:1412-23. [PMID: 33626250]  doi: 10.1056/NEJMoa2101765
4.
Abu-Raddad LJChemaitelly HButt AANational Study Group for COVID-19 Vaccination. Effectiveness of the BNT162b2 Covid-19 vaccine against the B.1.1.7 and B.1.351 variants [Letter]. N Engl J Med. 2021. [PMID: 33951357]  doi: 10.1056/NEJMc2104974
5.
Keehner JHorton LEPfeffer MAet al. SARS-CoV-2 infection after vaccination in health care workers in California [Letter]. N Engl J Med. 2021;384:1774-5. [PMID: 33755376]  doi: 10.1056/NEJMc2101927
6.
Daniel WNivet MWarner Jet al. Early evidence of the effect of SARS-CoV-2 vaccine at one medical center [Letter]. N Engl J Med. 2021;384:1962-3. [PMID: 33755374]  doi: 10.1056/NEJMc2102153
7.
Krammer FSrivastava KAlshammary Het al. Antibody responses in seropositive persons after a single dose of SARS-CoV-2 mRNA vaccine [Letter]. N Engl J Med. 2021;384:1372-4. [PMID: 33691060]  doi: 10.1056/NEJMc2101667
8.
National Center for Veterans Analysis and Statistics. Veteran Population. Accessed at www.va.gov/vetdata/veteran_population.asp on 4 April 2021.
9.
Agha ZLofgren RPVanRuiswyk JVet al. Are patients at Veterans Affairs medical centers sicker? A comparative analysis of health status and medical resource use. Arch Intern Med. 2000;160:3252-7. [PMID: 11088086]
10.
Butt AAYan PKapadia Set al. Social vulnerability in persons with chronic hepatitis C virus infection is associated with a higher risk of prescription opioid use. Sci Rep. 2021;11:5883. [PMID: 33723313]  doi: 10.1038/s41598-021-85283-6
11.
Justice ACErdos JBrandt Cet al. The Veterans Affairs Healthcare System: a unique laboratory for observational and interventional research. Med Care. 2006;44:S7-12. [PMID: 16849970]
12.
Charlson MEPompei PAles KLet al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis. 1987;40:373-83. [PMID: 3558716]
13.
Harrison SLFazio-Eynullayeva ELane DAet al. Comorbidities associated with mortality in 31,461 adults with COVID-19 in the United States: a federated electronic medical record analysis. PLoS Med. 2020;17:e1003321. [PMID: 32911500]  doi: 10.1371/journal.pmed.1003321
14.
Ioannou GNGreen PFan VSet al. Development of COVIDVax model to estimate the risk of SARS-CoV-2-related death among 7.6 million US veterans for use in vaccination prioritization. JAMA Netw Open. 2021;4:e214347. [PMID: 33822066]  doi: 10.1001/jamanetworkopen.2021.4347
15.
Jackson MLNelson JC. The test-negative design for estimating influenza vaccine effectiveness. Vaccine. 2013;31:2165-8. [PMID: 23499601]  doi: 10.1016/j.vaccine.2013.02.053
16.
Vandenbroucke JPBrickley EBVandenbroucke-Grauls CMJEet al. A test-negative design with additional population controls can be used to rapidly study causes of the SARS-CoV-2 epidemic. Epidemiology. 2020;31:836-43. [PMID: 32841988]  doi: 10.1097/EDE.0000000000001251
17.
Sullivan SGTchetgen Tchetgen EJCowling BJ. Theoretical basis of the test-negative study design for assessment of influenza vaccine effectiveness. Am J Epidemiol. 2016;184:345-53. [PMID: 27587721]  doi: 10.1093/aje/kww064
18.
Pearce N. Analysis of matched case–control studies. BMJ. 2016;352:i969. [PMID: 26916049]  doi: 10.1136/bmj.i969
19.
Austin PC. Type I error rates, coverage of confidence intervals, and variance estimation in propensity-score matched analyses. Int J Biostat. 2009;5:Article 13. [PMID: 20949126]  doi: 10.2202/1557-4679.1146
20.
Hightower AWOrenstein WAMartin SM. Recommendations for the use of Taylor series confidence intervals for estimates of vaccine efficacy. Bull World Health Organ. 1988;66:99-105. [PMID: 3260147]
21.
Sheppard KDSawyer PRitchie CSet al. Life-space mobility predicts nursing home admission over 6 years. J Aging Health. 2013;25:907-20. [PMID: 23965310]  doi: 10.1177/0898264313497507
22.
Brown CJFlood KL. Mobility limitation in the older patient: a clinical review. JAMA. 2013;310:1168-77. [PMID: 24045741]  doi: 10.1001/jama.2013.276566
23.
Kennedy RESawyer PWilliams CPet al. Life-space mobility change predicts 6-month mortality. J Am Geriatr Soc. 2017;65:833-8. [PMID: 28152168]  doi: 10.1111/jgs.14738

Comments

0 Comments
Sign In to Submit A Comment

Information & Authors

Information

Published In

cover image Annals of Internal Medicine
Annals of Internal Medicine
Volume 174Number 10October 2021
Pages: 1404 - 1408

History

Published online: 20 July 2021
Published in issue: October 2021

Keywords

Authors

Affiliations

Adeel A. Butt, MBBS, MS https://orcid.org/0000-0002-1118-1826
VA Pittsburgh Healthcare System, Pittsburgh, Pennsylvania, Weill Cornell Medical College, New York, New York, and Weill Cornell Medical College and Hamad Medical Corporation, Doha, Qatar (A.A.B.)
Saad B. Omer, MBBS, MPH, PhD
Yale School of Medicine, New Haven, Connecticut (S.B.O.)
Peng Yan, MS
VA Pittsburgh Healthcare System, Pittsburgh, Pennsylvania (P.Y.)
Obaid S. Shaikh, MD
VA Pittsburgh Healthcare System and University of Pittsburgh School of Medicine, Pittsburgh, Pennsylvania (O.S.S.)
Florian B. Mayr, MD, MPH https://orcid.org/0000-0002-2298-9011
VA Pittsburgh Healthcare System, University of Pittsburgh School of Medicine, and University of Pittsburgh, Pittsburgh, Pennsylvania (F.B.M.).
Note: Dr. Butt had complete access to the data at all times and accepts responsibility for the integrity of this article.
Disclaimer: This study used data created by the VA COVID-19 Shared Data Resource and resources and facilities of the VA Informatics and Computing Infrastructure, VA HSR RES 13-457. This material is the result of work that is also supported by resources and the use of facilities at the VA Pittsburgh Healthcare System and the central data repositories maintained by the VA Information Resource Center, including the Corporate Data Warehouse. The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the U.S. Department of Veterans Affairs or the funding agencies.
Financial Support: There was no external funding source for this study. Dr. Mayr is supported by K23GM132688 from the National Institutes of Health.
Reproducible Research Statement: Study protocol: Not available. Statistical code: Statistical code can be provided on request (e-mail, [email protected]). Data set: This study used data created and maintained by the Veterans Health Administration, U.S. Department of Veterans Affairs. These data are freely available to approved persons upon fulfillment of the specified requirements.
Corresponding Author: Adeel A. Butt, MBBS, MS, VA Pittsburgh Healthcare System, Building 30, Mailstop 151, University Drive C, Pittsburgh, PA 15240; e-mail, [email protected].
Current Author Addresses: Drs. Butt, Shaikh, and Mayr and Mr. Yan: Research Office Building, VA Pittsburgh Healthcare System, University Drive C, Pittsburgh, PA 15240.
Dr. Omer: Yale Institute for Global Health, 1 Church Street, New Haven, CT 06510.
Author Contributions: Conception and design: A.A. Butt, S.B. Omer.
Analysis and interpretation of the data: A.A. Butt, S.B. Omer, P. Yan, F.B. Mayr.
Drafting of the article: A.A. Butt, S.B. Omer.
Critical revision of the article for important intellectual content: A.A. Butt, S.B. Omer, O.S. Shaikh, F.B. Mayr.
Final approval of the article: A.A. Butt, S.B. Omer, P. Yan, O.S. Shaikh, F.B. Mayr.
Administrative, technical, or logistic support: A.A. Butt, O.S. Shaikh.
Collection and assembly of data: A.A. Butt, F.B. Mayr.
This article was published at Annals.org on 20 July 2021.

Metrics & Citations

Metrics

Citations

If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.

For more information or tips please see 'Downloading to a citation manager' in the Help menu.

Format





Download article citation data for:
Adeel A. Butt, Saad B. Omer, Peng Yan, et al. SARS-CoV-2 Vaccine Effectiveness in a High-Risk National Population in a Real-World Setting. Ann Intern Med.2021;174:1404-1408. [Epub 20 July 2021]. doi:10.7326/M21-1577

View More

Get Access

Login Options:
Purchase

You will be redirected to acponline.org to sign-in to Annals to complete your purchase.

Create your Free Account

You will be redirected to acponline.org to create an account that will provide access to Annals.

View options

PDF/ePub

View PDF/ePub

Media

Figures

Other

Tables

Share

Share

Copy the content Link

Share on social media