The COVID-19 Hospitalization Metric in the Pre- and Post-vaccination Eras as a Measure of Pandemic Severity: A Retrospective, Nationwide Cohort Study

Importance: Since the early days of the pandemic, COVID-19 hospitalizations have been used as a measure of pandemic severity. However, case definitions do not include assessments of disease severity, which may be impacted by prior vaccination.

Objective: To measure how the severity of respiratory disease changed among inpatients with documented SARS-CoV-2 infection and to measure the impact of vaccination status on these trends, in order to evaluate the accuracy of the metric of “hospitalization plus a positive SARS-CoV-2 test” for tracking pandemic severity.

Design: Retrospective cohort of inpatients with laboratory-confirmed SARS-CoV-2. All data were obtained from electronic health records.

Setting: Multi-center, nationwide study conducted in the healthcare system of the US Department of Veterans Affairs (VA) from March 1, 2020, through June 30, 2021.

Participants: All VA patients admitted to a VA hospital with a laboratory-confirmed SARS-CoV-2 infection within the 14-days prior to admission or during the hospital admission.

Main Outcome: Moderate-to-severe COVID-19 disease, defined by use of any supplemental oxygen or documented SpO2 <94%, during an inpatient hospitalization between one day before and two weeks after a positive SARS-CoV-2 test.

Exposure: SARS-CoV-2 vaccination status at the time of hospitalization. Patients were regarded as fully vaccinated starting 14 days after receiving the second of a 2-dose regimen or 14 days after receipt of a single-dose vaccine.

Results: Among 47,742 admissions in 38,508 unique patients with laboratory-confirmed SARS-CoV-2, N=28,731 met the criteria for moderate-to-severe COVID-19. The proportion with moderate-to-severe disease prior to widespread vaccine availability was 64.0% (95% CI, 63.1-64.9%) versus 52.0% in the later period (95% CI, 50.9-53.2%), p-value for non-constant effect, <0.001. Disease severity in the vaccine era among hospitalized patients was lower among both unvaccinated (55.0%, 95% CI, 53.7-56.4%) and vaccinated patients (42.6%, 95% CI, 40.6-44.8%).

Conclusions and Relevance: The proportion of hospitalizations that are due to severe COVID-19 has changed with vaccine availability, thus, increasing proportions of mild and asymptomatic cases are included in hospitalization reporting metrics. The addition of simple measures of disease severity to the case definition of a SARS-CoV-2 hospitalization is a straightforward and objective change that should improve the value of the metric for tracking SARS-CoV-2 disease burden.

Question: How well do COVID-19 hospitalization metrics correlate with severity of disease?

Findings: In a retrospective cohort of patients hospitalized and with a detected SARS-CoV-2 infection, the proportion of inpatients with moderate to severe disease at any time during their hospitalization as defined by any documentation of SpO2 <94% or receipt of any supplemental oxygen fell from 64% early in the pandemic to 52% (43% in vaccinated and 55% in unvaccinated patients) by the end of June 2021.

Meaning: Consideration should be given to updating the case definition of COVID-19 hospitalizations to better reflect cases of hospitalization caused by COVID-19 versus hospitalizations associated with detection of SARS-CoV-2. Documentation of SpO2 <94% is a simple and objective measure that could be incorporated into the surveillance definition to improve the value of the metric.

Since the early days of the pandemic, COVID-19 hospitalization rates have been used to track the community burden of SARS-CoV-2. The US Centers for Disease Control and Prevention (CDC)’s COVID-Net defines a COVID hospitalization as any patient admitted to the hospital within 14 days of a laboratory-confirmed diagnosis of SARS-CoV-2 infection, regardless of the reason for the admission.¹

Vaccination availability is expected to de-couple SARS-CoV-2 case rates from hospitalization rates, because vaccinated patients develop less severe disease and unvaccinated patients tend to be younger and healthier. Thus, metrics designed to determine the level of risk in a community, and in turn to inform decisions about restrictions and mitigation strategies, should increasingly focus on hospitalizations rather than cases. However, after vaccines became available in late 2020,^{2, 3} routine inpatient screening, common or mandated in many facilities, may identify incidental cases. If hospitalizations are used as a metric for policy decision-making, patients hospitalized for the management of COVID-19 disease should be distinguished from patients who are hospitalized and incidentally found to be infected with SARS-CoV-2.

The aim of this retrospective, national cohort study was to measure trends in severity of respiratory disease among inpatients with SARS-CoV-2 and how vaccination status has impacted these trends, with the goal of identifying simple, objective measures that could be used to improve the utility of the hospitalization metric.

All inpatient admissions to a Veterans Affairs (VA) hospital between 3/1/2020-6/30/2021 with a laboratory-confirmed diagnosis of SARS-CoV-2 up to 14 days prior to or during the admission were included, and data were extracted electronically from the VA Corporate Data Warehouse.

Moderate-to-severe COVID-19 was defined as any oxygen supplementation or any documented SpO2 < 94% recorded in the vital signs domain during a VA inpatient admission at any time between one day before and two weeks after the positive SARS-CoV-2 test. These criteria are adapted from the most stringent cut-off included in the National Institutes of Health (NIH) COVID-19 severity score and selected based on ability to reliably capture them from electronic health records.⁴ A sensitivity analysis was conducted using a clinical definition of respiratory distress (oxygen supplementation ≥ 2L NC or any documented SpO2 < 90%).

Admissions meeting criteria for moderate-to-severe disease were divided by the total numbers of admissions in the cohort over different calendar periods. The changing proportion of admissions with moderate-to-severe COVID-19 was modelled as a smooth function of time using a penalized spline in a Poisson regression and stratified by vaccination status.

Patients were considered fully vaccinated if ≥ 14 days had elapsed after receipt of a single dose of an adenovirus vaccine or after receipt of both doses of either mRNA-1273 (Moderna) or BNT162b2 (Pfizer) vaccines. As an additional assessment of severity of illness, the proportion of cases receiving treatment with dexamethasone was also evaluated as a function of time. A Pearson test of correlation between moderate-to-severe disease as determined by vital signs measures and dexamethasone receipt was performed.

The study was approved by the VA Boston Research and Development committee with a waiver for informed consent prior to data collection and analysis (protocol # 3328-X).

Among 47,742 admissions (32,546 before 1/21/2021, when the first patient in the cohort was fully vaccinated, and 15,196 after) representing 38,508 unique patients admitted with laboratory-confirmed SARS-CoV-2, 28,731 admissions had moderate-to-severe disease using vital signs criteria. The median age among the cohort was 71.1 (mean, 68.9, SD, 14.0); 28,901 (60.5%) were white; and 45,117 (94.5%) were male.

The proportion of inpatients with any documented SpO2 < 94% or oxygen supplementation changed over time (P < 0.001, Figure). From 3/1/2020-1/20/2021, the proportion of hospitalized patients with moderate-to-severe disease was 64.0% (95% CI, 63.1–64.9%) and from 1/21/2021-6/30/2021, was 52.0% (95% CI, 50.9–53.2%).

Among 15,196 admissions on or after 1/21/2021 (unvaccinated, 11,569; vaccinated, 3,627), 7,908 met the case definition for moderate-to-severe disease (unvaccinated, 6,362; vaccinated, 1,546). Among unvaccinated inpatients during this period, 55.0% (95% CI, 53.7–56.4%) had at least one documented SpO2 < 94%, versus 42.6% (95% CI, 40.6–44.8%) in vaccinated inpatients (Figure). Proportions of COVID-19 cases receiving treatment with dexamethasone also dropped in parallel with reduction in disease severity as measured by vital signs (Pearson correlation coefficient, 0.96, Figure), particularly in vaccinated patients.

Findings from the sensitivity analysis were similar. N = 14,175 admissions met the stricter case definition, including 32.3% (95% CI, 31.7–33.0%) of COVID-19 admissions prior to 1/21/2021 and 24.0% (95% CI, 23.3–24.8%) afterward (unvaccinated patients, 26.6%, 95% CI, 25.7–27.6% versus vaccinated, 15.8%, 95% CI, 14.5–17.1%, Supplementary Figure).

Prior to widespread vaccination, SARS-COV-2 cases were strongly correlated with hospitalizations, and both were reflective of COVID-19 disease burden and community risk. Since widespread vaccine availability, these metrics have become increasingly uncoupled, as reflected by the falling proportion of SARS-CoV-2 admissions with moderate-to-severe COVID-19. The uncoupling is more pronounced in fully vaccinated individuals, who continue to be strongly protected against severe disease. ^{5, 6} The changes in frequency of admissions with moderate-to-severe disease are strongly correlated with the changes in proportions of cases receiving dexamethasone, supporting an objective and reproducible approach that uses SpO2 values and supplemental oxygenation receipt to identify patients hospitalized for COVID-19. These findings suggest that reporting definitions may need to be revised to reflect the changing nature of the pandemic, particularly in regions with high levels of vaccine uptake.

With widespread vaccination, the current definition of COVID-19 hospitalizations includes progressively more mild or incidental diagnoses, for example, cases identified prior to surgery or prior to discharge, rather than hospitalizations due to severe COVID-19.⁷ Other recent studies similarly found that current definitions of “COVID-19 hospitalizations” combined with routine, and often mandatory, screening testing of all admissions may substantially overestimate the number of hospitalizations caused by SARS-CoV-2 infection.^{8, 9} In a pediatric population, 41% of reported admissions associated with SARS-CoV-2 infection were for reasons other than COVID-19, rates similar to those found when the simple definition of moderate to severe disease was applied in our cohort.⁹

Our study has several limitations. The VA population is not representative of the US population as a whole, having few women and no children. We did not control for patients with baseline hypoxemia or oxygen requirement (common among VA patients) or for altitude, and thus some mild cases may have been misclassified as moderate-to-severe. However, to ensure that severe cases are appropriately captured, a definition of high sensitivity is preferable to one favoring specificity. Cases of severe COVID-19 may have been admitted non-differentially to non-VA hospitals. Some Veterans may have been vaccinated outside of the VA, however, vaccines were typically available within the VA earlier than in the community. Finally, most of the data are from months before the more-transmissible delta variant became dominant. However, proportions of patients with moderate-to-severe respiratory distress or being treated with dexamethasone did not appear to be rising at the end of the observation period (6/30/2021), when delta was becoming predominant nationwide, suggesting stability of the vital signs metric for identifying moderate-to-severe COVID-19.

In this large, national cohort of US Veteran patients, the proportion of hospitalizations due to moderate-to-severe COVID-19 decreased following vaccine availability. Consideration should be given to updating definition of COVID-19 hospitalizations to improve differentiation between hospitalization caused by COVID-19 and those associated with detection of SARS-CoV-2 through the addition of straightforward and objective measures of disease severity.

Funding:

This work was supported by the VA Office of Research and Development, Cooperative Studies Program. 

Role of the Funder:

The funder had no role in the design of the study; the collection, analysis, and interpretation of the data; the writing of the manuscript, or the decision to submit the manuscript for publication.

Access to Data and Data Analysis:

Dr. Fillmore had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. 

Author Contributions:

Concept and design: WBE, NF, PM, SD

Acquisition, analysis, or interpretation of data: All authors

Drafting of the manuscript: WBE, PM, NF

Critical revision of the manuscript for important intellectual content: All authors

Statistical analysis: CZ, JL, NF

Competing Interest Declaration:

WBE and PM are site investigators for a study funded by Gilead pharmaceuticals (funds to institution). WBE is the recipient of VA HSRD funding. All other authors report no competing interests.

1. Centers for Disease Control and Prevention. Coronavirus Disease 2019 (COVID-19)-Associated Hospitalization Surveillance Network (COVID-NET). 2020.
2. Oliver SE, Gargano JW, Marin M, et al. The advisory committee on immunization practices’ interim recommendation for use of Pfizer-BioNTech COVID-19 vaccine—United States, December 2020. Morbidity and Mortality Weekly Report. 2020;69(50):1922.
3. Allen H, MSc AV, MFPH JF, MSc KA. Increased household transmission of COVID-19 cases associated with SARS-CoV-2 Variant of Concern B. 1.617. 2: a national casecontrol study. Public Health England. 2021;
4. National Institutes of Health. Clinical Spectrum of SARS-CoV-2 Infection. Updated 9/3/2021. Accessed September 6, 2021. https://www.covid19treatmentguidelines.nih.gov/overview/clinical-spectrum/
5. Centers for Disease Control and Prevention. COVID-19 vaccine breakthrough case investigation and reporting. 2021.
6. Lopez Bernal J, Andrews N, Gower C, et al. Effectiveness of Covid-19 Vaccines against the B.1.617.2 (Delta) Variant. New England Journal of Medicine. 2021;doi:10.1056/NEJMoa2108891
7. Calderwood MS, Deloney VM, Anderson DJ, et al. Policies and practices of SHEA Research Network hospitals during the COVID-19 pandemic. Infection control and hospital epidemiology. 2020;41(10):1127–1135. doi:10.1017/ice.2020.303
8. Webb NE, Osburn TS. Characteristics of Hospitalized Children Positive for SARS-CoV-2: Experience of a Large Center. Hospital Pediatrics. 2021;
9. Kushner LE, Schroeder AR, Kim J, Mathew R. “For COVID” or “With COVID”: Classification of SARS-CoV-2 Hospitalizations in Children. Hospital Pediatrics. 2021;