Real-World Effectiveness of BNT162b2 Against Infection and Severe Diseases in Children and AdolescentsFREE
Submit a Comment
Contributors must reveal any conflict of interest. Comments are moderated. Please see our information for authorsregarding comments on an Annals publication.
Visual Abstract. Real-World Effectiveness of BNT162b2 Against Infection and Severe Diseases in Children and Adolescents
Using electronic health record data from a national collaboration of pediatric health systems, this study examined the effectiveness of BNT162b2 vaccines for preventing COVID-19 infections and severe disease among previously uninfected children and adolescents. It found moderate effectiveness of the vaccine against the SARS-CoV-2 Omicron variant and high effectiveness against the Delta variant.
Abstract
Background:
The efficacy of the BNT162b2 vaccine in pediatrics was assessed by randomized trials before the Omicron variant’s emergence. The long-term durability of vaccine protection in this population during the Omicron period remains limited.
Objective:
To assess the effectiveness of BNT162b2 in preventing infection and severe diseases with various strains of the SARS-CoV-2 virus in previously uninfected children and adolescents.
Design:
Comparative effectiveness research accounting for underreported vaccination in 3 study cohorts: adolescents (12 to 20 years) during the Delta phase and children (5 to 11 years) and adolescents (12 to 20 years) during the Omicron phase.
Setting:
A national collaboration of pediatric health systems (PEDSnet).
Participants:
77 392 adolescents (45 007 vaccinated) during the Delta phase and 111 539 children (50 398 vaccinated) and 56 080 adolescents (21 180 vaccinated) during the Omicron phase.
Intervention:
First dose of the BNT162b2 vaccine versus no receipt of COVID-19 vaccine.
Measurements:
Outcomes of interest include documented infection, COVID-19 illness severity, admission to an intensive care unit (ICU), and cardiac complications. The effectiveness was reported as (1-relative risk)*100, with confounders balanced via propensity score stratification.
Results:
During the Delta period, the estimated effectiveness of the BNT162b2 vaccine was 98.4% (95% CI, 98.1% to 98.7%) against documented infection among adolescents, with no statistically significant waning after receipt of the first dose. An analysis of cardiac complications did not suggest a statistically significant difference between vaccinated and unvaccinated groups. During the Omicron period, the effectiveness against documented infection among children was estimated to be 74.3% (CI, 72.2% to 76.2%). Higher levels of effectiveness were seen against moderate or severe COVID-19 (75.5% [CI, 69.0% to 81.0%]) and ICU admission with COVID-19 (84.9% [CI, 64.8% to 93.5%]). Among adolescents, the effectiveness against documented Omicron infection was 85.5% (CI, 83.8% to 87.1%), with 84.8% (CI, 77.3% to 89.9%) against moderate or severe COVID-19, and 91.5% (CI, 69.5% to 97.6%) against ICU admission with COVID-19. The effectiveness of the BNT162b2 vaccine against the Omicron variant declined 4 months after the first dose and then stabilized. The analysis showed a lower risk for cardiac complications in the vaccinated group during the Omicron variant period.
Limitation:
Observational study design and potentially undocumented infection.
Conclusion:
This study suggests that BNT162b2 was effective for various COVID-19–related outcomes in children and adolescents during the Delta and Omicron periods, and there is some evidence of waning effectiveness over time.
Primary Funding Source:
National Institutes of Health.
The U.S. Food and Drug Administration expanded the emergency use authorization of the BNT162b2 messenger RNA COVID-19 vaccine (Pfizer–BioNTech) to 12 to 15 year olds on 10 May 2021 and to 5 to 11 year olds on 29 October 2021. As of 5 April 2023, the Centers for Disease Control and Prevention reports indicate that 40% of U.S. children aged 5 to 11 years and 72% of adolescents aged 12 to 18 years had received at least 1 dose of the vaccine. The emergence of the Omicron variant (B.1.1.529) and its subvariants in early 2022 led to a new surge in COVID-19 cases worldwide (1). The randomized trials of the BNT162b2 vaccine, which demonstrated high efficacy of 2 doses against COVID-19 (100% and 91% among those aged 12 to 15 years and 5 to 11 years, respectively), were done before the emergence of the Omicron variant (2, 3).
Several observational studies have been done to investigate the effectiveness of vaccination in real-world settings (4–8). However, prior studies have had limited follow-up periods, covering the Delta variant or earlier subvariants of Omicron periods only. Studies evaluating the Omicron variant have only assessed the short-term effects of the vaccine, with only 1 study involving children evaluating the effect beyond 3 months (9). There is limited information on the long-term durability of vaccine protection during the Omicron period. Few existing studies on U.S. pediatric populations have covered both hospitalized patients and those with mild or asymptomatic conditions. Furthermore, although studies have acknowledged limitations due to misclassification in vaccination status in real-world effectiveness studies, none have rigorously evaluated the effects of such misclassification nor accounted for the potential bias it may introduce.
To address these gaps in our knowledge of the pediatric effectiveness of SARS-CoV-2 vaccination, we designed this study to assess the real-world effectiveness of BNT162b2 among children and adolescents during the periods when the Delta and Omicron variants were predominant using electronic health record (EHR) data from a national network of U.S. pediatric medical centers. Our study used a novel comparative effectiveness research (CER) method and adjusted for underreporting issues in vaccination status and has several attractive features that strengthen the generalizability and reliability of our inference. First, it is the largest study to date in the United States estimating vaccine effectiveness in children and adolescents, covering a broad spectrum of the U.S. pediatric population. Second, the study examined the effectiveness against infection over a longer follow-up than any previous study, enabling evaluation of the durability of vaccine protection. Third, the study included a diverse representation of U.S. pediatric populations from primary care, specialty care, emergency department (ED), testing centers, and inpatient settings. Fourth, the study was the first to account for the incomplete capture of vaccination status by health systems in the United States. Finally, in addition to infection and severe diseases end points, we also studied the effect of vaccination on the incidence of myocarditis, pericarditis, or multisystem inflammatory syndrome (MIS) to assess the effect of vaccination relative to the potential risk for cardiac complications.
Methods
Data Sources
This study used EHR data from PEDSnet (10), which is a national collaboration of pediatric health systems that share EHR data, conduct research, and improve outcomes together. Participating institutions in this study included the Children’s Hospital of Philadelphia, Cincinnati Children’s Hospital Medical Center, Children’s Hospital Colorado, Ann & Robert H. Lurie Children’s Hospital of Chicago, Nationwide Children’s Hospital, Nemours Children’s Health System (inclusive of the Delaware and Florida health system), Seattle Children’s Hospital, and Stanford Children’s Health. Data were extracted from the PEDSnet COVID-19 Database Version Week 141 (11). A detailed description of EHR data is available in section 1 of the Supplement.
Specification of Hypothetical Trials and CER Studies
Hypothetical randomized controlled trials were specified to guide the design of observational studies to assess the real-world effectiveness of treatments (12). We designed and conducted CER studies to investigate the effectiveness of the BNT162b2 vaccine in preventing infection with various strains of the SARS-CoV-2 virus in children and adolescents in the United States. The 3 study cohorts focused on documented SARS-CoV-2 infection and outcomes: study cohort 1 (Delta study in adolescents): adolescents aged 12 to 20 years during the period when the Delta variant was prevalent from 1 July to 30 November 2021; study cohort 2 (Omicron study in children): children aged 5 to 11 years during the period when the Omicron variant was prevalent from 1 January to 30 November 2022; and study cohort 3 (Omicron study in adolescents): adolescents aged 12 to 20 years during the period when the Omicron variant was prevalent from 1 January to 30 November 2022.
The design of hypothetical trials and implementation procedures in real-world data are summarized in Supplement Table 1.
Eligibility criteria included age of 5 to 11 years for children or 12 to 20 years for adolescents at the start of the study period and no previous COVID-19 vaccination or documented SARS-CoV-2 infection. In addition, participants were required to have a prior encounter (including telephone or telehealth encounters) within 18 months of cohort entrance to ensure that they had an ongoing interaction with the health system.
The intervention of interest was vaccination, in comparison with no receipt of any type of COVID-19 vaccine. Given that the BNT162b2 vaccine covered more than 85% of documented vaccinations among children and adolescents in the PEDSnet database, in this study we focused primarily on studying the effectiveness of this vaccine, although the Supplement reports a sensitivity analysis investigating all types of reported COVID-19 vaccines assessed in the United States, with 85.7% BNT162b2, 1.9% mRNA-1273 (Moderna), and 12.3% unspecified COVID-19 vaccine.
In this CER study using real-world data, the cohort entrance date for the intervention group was defined as the date of the first dose of the BNT162b2 vaccine, whereas for the comparator group, it was a randomly selected date from visits, chosen to ensure that the distribution of index dates for the control group matched the distribution of index dates for the vaccination group to control for time effects. The risk period for the study began 28 days after the index date, such that infections within 28 days were excluded.
Randomized trials achieve balance across potential confounders by randomly allocating the treatment to intervention and comparator groups. In our study, we attempted to balance the intervention and comparator groups by adjusting for a large number of measured confounders collected before cohort entry using propensity score stratification (13). We built the propensity score model on the basis of demographic factors, including age, sex, race, and ethnicity; clinical factors, including obesity status, a baseline chronic condition indicator as defined by the Pediatric Medical Complexity Algorithm (14), and a list of preexisting chronic conditions; and health care utilization factors collected before the cohort entry, including the number of inpatients, outpatients, ED visits, unique mediations, and negative COVID-19 tests. We stratified the patients into propensity score quintiles on the basis of these factors. See Supplement Table 3 for detailed definitions of study variables.
The 4 COVID-19 outcomes of interest were documented SARS-CoV-2 infection, mild COVID-19, moderate or severe COVID-19, and ICU admission with COVID-19. We did not evaluate death from COVID-19 because it was too rare among children and adolescents to study quantitatively. In our study, SARS-CoV-2 infections were defined by positive polymerase chain reaction, serology, or antigen tests or diagnoses of COVID-19 or by occurrence of postacute sequelae of SARS-CoV-2 or MIS in children regardless of the presence of symptoms. Classification of mild, moderate, or severe COVID-19 illness was defined on the basis of the symptoms and health conditions diagnosed from 7 days before to 13 days after the date of a documented COVID-19 infection, as in Forrest and colleagues study (15). Admission to an ICU with COVID-19 was defined by any ICU visit 7 days before and 13 days after documented SARS-CoV-2 infection. In addition, we considered the clinical outcomes of cardiac complications identified as incidence of myocarditis, pericarditis, or MIS to allow for a comprehensive capture of potential cardiac complications after infection and evaluate the effect of the BNT162b2 vaccine in terms of cardiac risks.
Statistical Analysis
We evaluated covariate balancing after propensity score stratification by plotting the standardized mean differences (SMDs) between variable values for the vaccinated and unvaccinated groups, with a difference of 0.1 or less indicating an acceptable balance. We used Poisson regression to estimate the relative risk between 2 treatment groups for the risk for each outcome while adjusting for different follow-up lengths among participants. Because immunization records are often captured and stored across multiple disconnected sources, resulting in underreported vaccination records in patients’ EHRs, we mitigated the potential bias arising from this underreporting issue by incorporating an integration likelihood (16, 17) of the Poisson regression with a prespecified range of misclassification rates. The vaccine effectiveness was defined as (1-relative risk)*100. The details of statistical methods are described in section 2 of the Supplement.
We conducted secondary analyses stratified by 2-month intervals since receipt of vaccination to investigate the durability of vaccine protection. Subgroup analyses were also done to investigate differences in vaccine protection according to age groups (5 to 8, 9 to 11, 12 to 15, and 16 to 20 years).
Sensitivity Analyses
Extensive sensitivity analyses were done to evaluate the robustness of the research findings; see sections 6 to 15 and section 19 of the Supplement for the effects of the cohort design. In scenarios in which any categorical covariates were unbalanced (with an SMD >0.1), we included a sensitivity analysis excluding participants in that category. A sensitivity analysis defining the risk period with no waiting period after the index date was done. To assess whether population heterogeneity, established by applying eligibility criteria that required a prior encounter within 18 months of cohort entry, could influence our vaccine effectiveness estimates, we conducted sensitivity analyses within a more restricted time window. We conducted a dose-response study of BNT162b2 by defining a single-dose vaccination as 1 dose 28 days before infection and a 2-dose vaccination as the receipt of a second dose 7 days before infection. A sensitivity analysis was done to compare the proposed method to a sequential target trial emulation pipeline not accounting for underreporting issues to assess the robustness of findings (18). Because the proportion of patients entered with ED visits is relatively low in the vaccinated cohort compared with the unvaccinated, we conducted a sensitivity analysis excluding all participants who entered the cohort during an ED visit. Residual study bias from unmeasured and systematic sources can still exist in observational studies after controlling for measured confounders; thus, we conducted negative control outcome experiments (13, 19, 20) in which the null hypothesis of no effect was believed to be true using 40 negative control outcomes prespecified by pediatric physicians. The empirical null distribution and calibrated effectiveness were reported as sensitivity analyses. The relative risks for cardiac complications defined by myocarditis or pericarditis (excluding MIS) were estimated. We also reported the estimated vaccine effectiveness from all brands of COVID-19 vaccines. Given the prolonged presence of the Omicron variant and the emergence of several subvariants, we conducted a secondary analysis to assess the vaccine effectiveness related to subvariants of Omicron.
Missingness in Vaccine Records
Vaccine status may be missing for persons whose vaccine doses were administered by a site outside of the PEDSnet network care delivery sites. It is likely that patients recorded as vaccinated in the EHR are true positives, so specificity could be very high, but sensitivity would be reduced by undocumented vaccinations (false negatives). To account for potential bias from the underreporting issue in vaccination status, a range of possible sensitivities based on our prior study was prespecified for each study. The sensitivity range was considered to be 0.8 to 1 for the study involving children and 0.7 to 0.9 for the studies involving adolescents. By accounting for the underreporting and specifying a range of sensitivity, the study aimed to minimize the effect of bias caused by the underreporting in the estimation of the effectiveness of the COVID-19 vaccine among children and adolescents. The details of statistical methods are described in section 2 of the Supplement.
To further evaluate the robustness of statistical methods we used to account for underreporting, we varied our CER method by considering alternative methods for bias correction, including the naive method (without adjusting for underreporting), using different ranges of misclassification rates, and using a fully Bayesian method (21). To evaluate the effect of differential misclassification on effectiveness estimates, we conducted sensitivity analyses simulating vaccination status according to various differential misclassification scenarios. Results from these sensitivity analyses are summarized in sections 16 to 18 of the Supplement.
Role of the Funding Source
The funders had no role in the design or conduct of the study; collection, management, analysis, or interpretation of the data; approval of the manuscript; or the decision to submit the manuscript for publication.
Results
Study Population
Figure 1 shows the process of selection of 3 study cohorts. A total of 77 392 adolescents (45 007 vaccinated) within the PEDSnet network were identified to study the effectiveness of vaccination against infection with the Delta variant and severe outcomes (see Table 1 for baseline characteristics). A total of 111 539 children (50 398 vaccinated) and 56 080 adolescents (21 180 vaccinated) were included in the cohort to study the effectiveness of vaccination against infection with the Omicron variant (see Table 2 for baseline characteristics). The vaccinated and unvaccinated groups had a slightly unbalanced distribution of testing rates before cohort entry across all 3 cohorts. After propensity score stratification, all covariates were well balanced between vaccinated and unvaccinated groups with an SMD smaller than 0.1 in the Omicron study involving children (Supplement Figure 12) and involving adolescents (Supplement Figure 13). In the study evaluating vaccine effectiveness for adolescents during the Delta period, 1 site remained unbalanced after propensity score stratification with an SMD larger than 0.1, and thus a sensitivity analysis was done by excluding participants from this site, which gave consistent results with the primary analysis (see Supplement Figure 11 and section 6 in the Supplement). Additional characteristics of the study cohorts, including additional medical conditions, deaths, and follow-up durations, are summarized in section 5 of the Supplement.
| Characteristic | Vaccinated (n = 45 007) | Unvaccinated (n = 32 385) | Overall (n = 77 392) |
|---|---|---|---|
| Median age (Q1–Q3), y | 14 (13–16) | 15 (13–17) | 15 (13–16) |
| Distribution, n (%) | |||
| 12 y | 8922 (19.8) | 4926 (15.2) | 13 848 (17.9) |
| 13 y | 8099 (18.0) | 4864 (15.0) | 12 963 (16.7) |
| 14 y | 8037 (17.9) | 4923 (15.2) | 12 960 (16.7) |
| 15 y | 7311 (16.2) | 4749 (14.7) | 12 060 (15.6) |
| 16 y | 5450 (12.1) | 4546 (14.0) | 9996 (12.9) |
| 17 y | 4076 (9.1) | 3991 (12.3) | 8067 (10.4) |
| 18 y | 1674 (3.7) | 2075 (6.4) | 3749 (4.8) |
| 19 y | 942 (2.1) | 1461 (4.5) | 2403 (3.1) |
| 20 y | 496 (1.1) | 850 (2.6) | 1346 (1.7) |
| Sex, n (%) | |||
| Female | 23 589 (52.4) | 16 500 (50.9) | 40 089 (51.8) |
| Male | 21 416 (47.6) | 15 880 (49.0) | 37 296 (48.2) |
| Ethnicity, n (%) | |||
| White | 16 446 (50.8) | 17 964 (39.9) | 34 410 (44.5) |
| Black/African American | 6019 (18.6) | 12 012 (26.7) | 18 031 (23.3) |
| Hispanic | 4925 (15.2) | 9629 (21.4) | 14 554 (18.8) |
| Other/unknown | 4995 (15.4) | 5402 (12.0) | 10 397 (13.4) |
| Hospital, n (%) | |||
| A | 5424 (12.1) | 7385 (22.8) | 12 809 (16.6) |
| B | 12 884 (28.6) | 5216 (16.1) | 18 100 (23.4) |
| C | 6333 (14.1) | 3457 (10.7) | 9790 (12.6) |
| D | 1723 (3.8) | 914 (2.8) | 2637 (3.4) |
| E | 4369 (9.7) | 6063 (18.7) | 10 432 (13.5) |
| F | 12 831 (28.5) | 3409 (10.5) | 16 240 (21.0) |
| G | 1430 (3.2) | 1457 (4.5) | 2887 (3.7) |
| H | 13 (0.0) | 4484 (13.8) | 4497 (5.8) |
| Entry time, n (%) | |||
| July–September 2021 | 38 335 (85.2) | 24 509 (75.7) | 62 844 (81.2) |
| October–November 2021 | 6672 (14.8) | 7876 (24.3) | 14 548 (18.8) |
| Obesity, n (%) | |||
| 0 | 28 029 (62.3) | 22 479 (69.4) | 50 508 (65.3) |
| 1 | 16 978 (37.7) | 9906 (30.6) | 26 884 (34.7) |
| Pediatric Medical Complexity Algorithm, n (%) | |||
| 0 | 25 634 (57.0) | 19 916 (61.5) | 45 550 (58.9) |
| 1 | 10 915 (24.3) | 6417 (19.8) | 17 332 (22.4) |
| 2 | 8458 (18.8) | 6052 (18.7) | 14 510 (18.7) |
| Negative tests before entry, n (%) | |||
| 0 | 1330 (3.0) | 2888 (8.9) | 4218 (5.5) |
| 1 | 34 272 (76.1) | 16 299 (50.3) | 50 571 (65.3) |
| 2 | 7388 (16.4) | 9739 (30.1) | 17 127 (22.1) |
| ≥3 | 2017 (4.5) | 3459 (10.7) | 5476 (7.1) |
Q = quartile.
| Characteristic | Omicron Study in Children | Omicron Study in Adolescents | ||||
|---|---|---|---|---|---|---|
| Vaccinated (n = 50 398) | Unvaccinated (n = 61 141) | Overall (n = 111 539) | Vaccinated (n = 21 180) | Unvaccinated (n = 34 900) | Overall (n = 56 080) | |
| Median age (Q1–Q3), y | 8 (6–10) | 7 (6–9) | 8 (6–10) | 14 (13–16) | 15 (13–17) | 15 (13–17) |
| Distribution, n (%) | ||||||
| 5 y | 8165 (16.2) | 13 321 (21.8) | 21 486 (19.3) | – | – | – |
| 6 y | 7447 (14.8) | 11 314 (18.5) | 18 761 (16.8) | – | – | – |
| 7 y | 7090 (14.1) | 9151 (15.0) | 16 241 (14.6) | – | – | – |
| 8 y | 7028 (13.9) | 7922 (13.0) | 14 950 (13.4) | – | – | – |
| 9 y | 6773 (13.4) | 7085 (11.6) | 13 858 (12.4) | – | – | – |
| 10 y | 7011 (13.9) | 6434 (10.5) | 13 445 (12.1) | – | – | – |
| 11 y | 6884 (13.7) | 5914 (9.7) | 12 798 (11.5) | – | – | – |
| 12 y | – | – | – | 4754 (22.4) | 5760 (16.5) | 10 514 (18.7) |
| 13 y | – | – | – | 3421 (16.2) | 5520 (15.8) | 8941 (15.9) |
| 14 y | – | – | – | 3338 (15.8) | 5299 (15.2) | 8637 (15.4) |
| 15 y | – | – | – | 3123 (14.7) | 5315 (15.2) | 8438 (15.0) |
| 16 y | – | – | – | 2634 (12.4) | 4944 (14.2) | 7578 (13.5) |
| 17 y | – | – | – | 2201 (10.4) | 3836 (11.0) | 6037 (10.8) |
| 18 y | – | – | – | 986 (4.7) | 2114 (6.1) | 3100 (5.5) |
| 19 y | – | – | – | 475 (2.2) | 1400 (4.0) | 1875 (3.3) |
| 20 y | – | – | – | 248 (1.2) | 712 (2.0) | 960 (1.7) |
| Sex, n (%) | ||||||
| Female | 23 962 (47.5) | 28 669 (46.9) | 52 631 (47.2) | 11 402 (53.8) | 17 954 (51.4) | 29 356 (52.3) |
| Male | 26 436 (52.5) | 32 468 (53.1) | 58 904 (52.8) | 9775 (46.2) | 16 939 (48.5) | 26 714 (47.6) |
| Ethnicity, n (%) | ||||||
| White | 14 399 (28.6) | 24 644 (40.3) | 39 043 (35.0) | 16 240 (46.5) | 6836 (32.3) | 23 076 (41.1) |
| Black/African American | 13 711 (27.2) | 13 733 (22.5) | 27 444 (24.6) | 6154 (17.6) | 6157 (29.1) | 12 311 (22.0) |
| Hispanic | 12 119 (24.0) | 12 781 (20.9) | 24 900 (22.3) | 6287 (18.0) | 3784 (17.9) | 10 071 (18.0) |
| Other/unknown | 10 169 (20.2) | 9983 (16.3) | 20 152 (18.1) | 6219 (17.8) | 4403 (20.8) | 10 622 (18.9) |
| Hospital, n (%) | ||||||
| A | 5019 (10.0) | 9266 (15.2) | 14 285 (12.8) | 2131 (10.1) | 5183 (14.9) | 7314 (13.0) |
| B | 15 229 (30.2) | 13 168 (21.5) | 28 397 (25.5) | 6397 (30.2) | 6556 (18.8) | 12 953 (23.1) |
| C | 5482 (10.9) | 7409 (12.1) | 12 891 (11.6) | 1719 (8.1) | 4075 (11.7) | 5794 (10.3) |
| D | 4766 (9.5) | 2878 (4.7) | 7644 (6.9) | 678 (3.2) | 1337 (3.8) | 2015 (3.6) |
| E | 5843 (11.6) | 10 551 (17.3) | 16 394 (14.7) | 2047 (9.7) | 4263 (12.2) | 6310 (11.3) |
| F | 9786 (19.4) | 11 348 (18.6) | 21 134 (18.9) | 3563 (16.8) | 5186 (14.9) | 8749 (15.6) |
| G | 1250 (2.5) | 3239 (5.3) | 4489 (4.0) | 622 (2.9) | 2353 (6.7) | 2975 (5.3) |
| H | 3023 (6.0) | 3282 (5.4) | 6305 (5.7) | 4023 (19.0) | 5947 (17.0) | 9970 (17.8) |
| Entry time, n (%) | ||||||
| January–March 2022 | 37 970 (75.3) | 32 523 (53.2) | 70 493 (63.2) | 14 684 (69.3) | 19 032 (54.5) | 33 716 (60.1) |
| April–June 2022 | 5882 (11.7) | 11 919 (19.5) | 17 801 (16.0) | 3344 (15.8) | 7087 (20.3) | 10 431 (18.6) |
| July–September 2022 | 4994 (9.9) | 10 329 (16.9) | 15 323 (13.7) | 2206 (10.4) | 5479 (15.7) | 7685 (13.7) |
| October–November 2022 | 1552 (3.1) | 6370 (10.4) | 7922 (7.1) | 946 (4.5) | 3302 (9.5) | 4248 (7.6) |
| Obesity, n (%) | ||||||
| 0 | 33 381 (66.2) | 42 165 (69.0) | 75 546 (67.7) | 13 832 (65.3) | 23 895 (68.5) | 37 727 (67.3) |
| 1 | 17 017 (33.8) | 18 976 (31.0) | 35 993 (32.3) | 7348 (34.7) | 11 005 (31.5) | 18 353 (32.7) |
| Pediatric Medical Complexity Algorithm, n (%) | ||||||
| 0 | 33 870 (67.2) | 40 976 (67.0) | 74 846 (67.1) | 13 482 (63.7%) | 21 079 (60.4) | 34 561 (61.6) |
| 1 | 10 000 (19.8) | 11 189 (18.3) | 21 189 (19.0) | 4382 (20.7%) | 6764 (19.4) | 11 146 (19.9) |
| 2 | 6528 (13.0) | 8976 (14.7) | 15 504 (13.9) | 3316 (15.7%) | 7057 (20.2) | 10 373 (18.5) |
| Negative tests before entry, n (%) | ||||||
| 0 | 2337 (4.6) | 5640 (9.2) | 7977 (7.2) | 768 (3.6) | 2966 (8.5) | 3734 (6.7) |
| 1 | 34 077 (67.6) | 28 417 (46.5) | 62 494 (56.0) | 15 707 (74.2) | 17 303 (49.6) | 33 010 (58.9) |
| 2 | 10 514 (20.9) | 19 816 (32.4) | 30 330 (27.2) | 3654 (17.3) | 11 012 (31.6) | 14 666 (26.2) |
| ≥3 | 3470 (6.9) | 7268 (11.9) | 10 738 (9.6) | 1051 (5.0) | 3619 (10.4) | 4670 (8.3) |
Q = quartile.
Vaccine Effectiveness
Table 3 summarizes the estimated vaccine effectiveness in 3 study cohorts, and Figure 2 shows the durability of protection. The vaccine effectiveness was estimated to be 98.4% (95% CI, 98.1% to 98.7%) among adolescents in the Delta period, 74.3% (CI, 72.2% to 76.2%) against documented infection among children in the Omicron period, and 85.5% (CI, 83.8% to 87.1%) among adolescents in the Omicron period. During the Delta period, the vaccine effectiveness against documented infection remained stable throughout the follow-up period of the study. Four months after the first dose, vaccine effectiveness against documented infection with Omicron declined from 82.3% (CI, 77.9% to 85.8%) to 70.6% (CI, 65.9% to 74.6%) among children, and from 91.3% (CI, 87.6% to 94.0%) to 82.9% (CI, 79.0% to 86.1%) among adolescents. Although vaccine effectiveness against documented infection stabilized after this initial decline, the corresponding CIs were much wider, indicating higher levels of uncertainty.
| Epidemiologic Measure | Vaccinated | Unvaccinated | Overall | Vaccine Effectiveness (95% CI), % |
|---|---|---|---|---|
| Delta study in adolescents | ||||
| Follow-up | ||||
| Total follow-up, n of person-weeks | 644 162 | 398 906 | 1 043 068 | – |
| Median (Q1–Q3) | 16 (12–18) | 13 (9–17) | 15 (10–18) | – |
| Absolute risk, % | ||||
| Documented infection | 0.35 | 5.26 | 2.41 | 98.4 (98.1–98.7) |
| Mild COVID-19 | 0.06 | 1.43 | 0.63 | 99.0 (98.5–99.3) |
| Moderate or severe COVID-19 | 0.03 | 0.49 | 0.22 | 98.7 (97.4–99.3) |
| ICU admission with COVID-19 | ≤0.01 | 0.05 | ≤0.03 | 99.0 (92.5–99.9) |
| Cardiac complication | 0.02 | 0.03 | 0.02 | 1.22 (0.34–4.35)* |
| Age 12–15 y | ||||
| Total follow-up, n of person-weeks | 458 981 | 229 083 | 688 064 | – |
| Documented infection, % | 0.34 | 5.52 | 2.28 | 99.0 (98.6–99.3) |
| Age 16–21 y | ||||
| Total follow-up, n of person-weeks | 185 181 | 169 823 | 355 003 | – |
| Documented infection, % | 0.36 | 4.91 | 2.64 | 97.0 (95.9–97.8) |
| Omicron study in children | ||||
| Follow-up | ||||
| Total follow-up, n of person-weeks | 1 925 686 | 1 911 599 | 3 837 285 | – |
| Median (Q1–Q3) | 44 (35–46) | 36 (19–44) | 40 (25–45) | – |
| Absolute risk, % | ||||
| Documented infection | 1.89 | 5.46 | 3.85 | 74.3 (72.2–76.2) |
| Mild COVID-19 | 0.54 | 1.55 | 1.09 | 73.5 (69.2–77.1) |
| Moderate or severe COVID-19 | 0.19 | 0.67 | 0.45 | 75.5 (69.0–81.0) |
| ICU admission with COVID-19 | 0.02 | 0.08 | 0.05 | 84.9 (64.8–93.5) |
| Cardiac complication | ≤0.01 | 0.03 | ≤0.02 | 0.28 (0.08–0.95)* |
| Age 5–8 y | ||||
| Total follow-up, n of person-weeks | 1 101 418 | 1 254 819 | 2 356 236 | – |
| Documented infection, % | 1.96 | 5.10 | 3.78 | 71.3 (68.2–74.1) |
| Age 9–11 y | ||||
| Total follow-up, n of person-weeks | 824 268 | 656 780 | 1 481 049 | – |
| Documented infection, % | 1.80 | 6.11 | 3.95 | 77.9 (75.1–80.4) |
| Omicron study in adolescents | ||||
| Follow-up | ||||
| Total follow-up, n of person-weeks | 772 176 | 1 113 561 | 1 885 736 | – |
| Median (Q1–Q3) | 42 (30–45) | 37 (22–44) | 39 (25–45) | – |
| Absolute risk, % | ||||
| Documented infection | 1.82 | 8.17 | 5.77 | 85.5 (83.8–87.1) |
| Mild COVID-19 | 0.43 | 2.07 | 1.45 | 87.0 (83.5–89.8) |
| Moderate or severe COVID-19 | 0.15 | 0.88 | 0.60 | 84.8 (77.3–89.9) |
| ICU admission with COVID-19 | ≤0.02 | 0.14 | ≤0.10 | 91.5 (69.5–97.6) |
| Cardiac complication | ≤0.02 | 0.05 | ≤0.04 | 0.10 (0.02–0.57)* |
| Age 12–15 y | ||||
| Total follow-up, n of person-weeks | 524 053 | 654 315 | 1 178 368 | – |
| Documented infection, % | 1.93 | 8.24 | 5.67 | 85.8 (83.6–87.7) |
| Age 16–21 y | ||||
| Total follow-up, n of person-weeks | 248 123 | 459 246 | 707 368 | – |
| Documented infection, % | 1.60 | 8.07 | 5.93 | 85.9 (82.7–88.5) |
ICU = intensive care unit; Q = quartile.
* Relative risk in the vaccinated groups compared with the unvaccinated.
Vaccine Effectiveness: Severe Illness and Complications
During the Delta period, the vaccine was found to have high effectiveness against severe infections. The estimated relative risk of the vaccine on cardiac complications was 1.22 (CI, 0.34 to 4.35). The estimated vaccine effectiveness against the Omicron variant in children was 73.5% (CI, 69.2% to 77.1%) against mild COVID-19, 75.5% (CI, 69.0% to 81.0%) against moderate or severe COVID-19, and 84.9% (CI, 64.8% to 93.5%) against ICU admission with COVID-19. The estimated relative risk of the vaccine on cardiac complications was 0.28 (CI, 0.08 to 0.95). In the Omicron study in adolescents, the vaccine effectiveness was estimated to be 87.0% (CI, 83.5% to 89.8%) against mild COVID-19, 84.8% (CI, 77.3% to 89.9%) against moderate or severe COVID-19, and 91.5% (CI, 69.5% to 97.6%) against ICU admission with COVID-19. The estimated relative risk of the vaccine on cardiac complications was 0.10 (CI, 0.02 to 0.57) in this cohort.
Sensitivity Analyses and Addressing Misclassification Bias
Section 9 of the Supplement presents the vaccine effectiveness against various Omicron subvariants. The effectiveness against subvariants BA.1, BA.2, BA.4, and BA.5 aligns with our primary findings, whereas the vaccine effectiveness seems to be lower against the BQ.1, XBB, and subsequent subvariants. Section 11 of the Supplement presents the results for the dose-response analysis. Section 14 of the Supplement presents negative control experiments of 3 study cohorts using 40 negative control outcomes. After accounting for systematic error through calibration using negative control outcomes, our findings indicate a slight shift in point estimates accompanied by wider CIs. This suggests the presence of a minor degree of systematic error, as well as additional uncertainty characterized by the estimated distribution derived from the negative control outcomes. Section 19 of the Supplement summarizes the comparative results to a sequential target trial emulation not accounting for underreporting issues of vaccination, which indicated reasonably consistent findings.
Section 16 of the Supplement shows effectiveness estimated from the naive method and proposed CER method with different ranges of sensitivity of vaccination status captured by EHR. The comparison results indicated that the vaccine effectiveness was reasonably consistent across different sensitivity ranges, suggesting that our primary analysis was robust to changes in the range of sensitivity considered. Section 17 of the Supplement shows the comparative results to a fully Bayesian method indicating nearly identical results. Section 18 of the Supplement shows sensitivity analyses on differential misclassifications, which shows that the novel CER method corrects the bias even when the nondifferential misclassification assumption does not hold.
Discussion
We estimated the effectiveness of BNT162b2 vaccines for the prevention of documented SARS-CoV-2 infections and severe diseases in a national network of pediatric health systems in the United States for 3 study cohorts. During the period when the Delta variant was dominant, the BNT162b2 vaccine was associated with strong protection in adolescents, with effectiveness higher than 95%, and with little evidence of waning during the follow-up period. Our findings against the Delta infection among adolescents are consistent with vaccine efficacy seen in the BNT162b2 clinical trial involving adolescents aged between 12 and 15 years, which demonstrated vaccine efficacy of 100% (CI, 75.3% to 100%) against documented SARS-CoV-2 infection (2). Our estimates of vaccine effectiveness against severe diseases are consistent with a case–control study based on test-negative design, which found an effectiveness of 94% (CI, 90% to 96%) against hospitalization and 98% (CI, 93% to 99%) against ICU admission (7).
In our study, during the period when the Omicron variant was dominant, the estimated vaccine effectiveness was approximately 70% for children and 85% for adolescents. The estimated protection decreased by roughly 10% around 4 months from the first dose and slightly waned over time. Previous studies have shown vaccine effectiveness against Omicron infection, ranging from 20% to 80% among children and adolescents. An analysis using a test-negative design in Scotland during the Omicron period found a vaccine effectiveness of 81.2% (CI, 77.7% to 84.2%) for adolescents aged 12 to 15 years and 65.5% (CI, 56.0% to 73.0%) for those aged 16 to 17 years 2 to 5 weeks after full vaccination. This effectiveness decreased to 43.3% (CI, 30.0% to 54.2%) and 8.9% (CI, −19.1% to 30.3%) after 10 to 13 weeks, respectively (22). With a test-negative design, data from U.S. pharmacy-based, drive-through SARS-CoV-2 testing sites indicated that the estimated vaccine effectiveness for children aged 5 to 11 years was 60.1% within 2 to 4 weeks after the second dose and declined to 28.9% during the second month after vaccination. For adolescents aged 12 to 15 years, the effectiveness was 59.5% within 2 to 4 weeks after the second dose and decreased to 16.6% during the subsequent month (23). Another test-negative design analysis in the United States indicated that in children aged 5 to 11 years, the effectiveness was 59.9% (CI, 58.5% to 61.2%) at 1 month, 33.7% (CI, 32.6% to 34.8%) at 4 months, and 14.9% (CI, 12.3% to 17.5%) at 10 months after the first dose (24). A retrospective study among children aged 5 to 11 years in Italy found vaccine effectiveness to be 29.4% (CI, 28.5% to 30.2%) against documented infection and 41.1% (CI, 22.2% to 55.4%) against severe COVID-19 (8). A Singapore study found that in fully vaccinated children aged 5 to 11 years, vaccine effectiveness was 65.3% (CI, 62.0% to 68.3%) against polymerase chain reaction–confirmed SARS-CoV-2 infection and 82.7% (CI, 74.8% to 88.2%) against COVID-19–related hospitalization (23). In addition, we found that vaccine effectiveness is substantially lower against the later subvariants of Omicron, including BQ.1 and XBB. However, it remains unclear whether this is a true subvariant effect or evidence of further waning over time. Continued research is desirable to understand vaccine effectiveness on future subvariants and its potential waning effects.
This study did not identify a statistically significant elevated risk for cardiac complications among vaccinated adolescents during the Delta variant period, and it even demonstrated a lower risk in the vaccinated group during the Omicron variant period. This finding may seem unexpected given that cases of myocarditis and pericarditis after messenger RNA COVID-19 vaccines received substantial attention (26–28). However, it is essential to note that previous studies indicated a much higher risk for myocarditis or pericarditis after a documented SARS-CoV-2 infection in the pediatric population (29), with an article finding 36.8 times higher risk (CI, 25.0 to 48.6) in persons younger than 16 years after COVID-19 (30), and that myocarditis is a common symptom for patients diagnosed with MIS (31, 32). It is possible that cases of myocarditis, pericarditis, or MIS occurred after undocumented SARS-CoV-2 infections. For example, 31.5% of occurrences of MIS did not have SARS-CoV-2 infections documented in the PEDSnet EHR database. Also note that during this period, especially the Omicron period, positive tests may have been from at-home tests or otherwise outside the system. Further assessment of cardiac complications after vaccination and COVID-19 is warranted to help provide a more complete picture of risk or benefit during a changing pandemic.
Our study has several strengths. First, we used a national network of academic medical centers that covered a diverse cohort being more representative of the general pediatric population, provided a robust sample size, and allowed for multiple subgroup analyses and detection of rare outcomes. Second, the richness of these EHR data allowed us to investigate the effectiveness against infection of different levels of severity as well as adjust for a broad set of confounders. Third, we conducted the negative control outcome experiments to assess the potential residual bias due to unmeasured confounders and other potential sources of systematic bias in the data. These experiments revealed a small amount of systematic error but with excessive uncertainty across different negative control outcomes, leading to wider CIs of our estimated effectiveness that honestly reflect the effects of unmeasured confounding and other potential sources of residual biases (33). Finally, to the best of our knowledge, this is the first real-world effectiveness study evaluating COVID-19 vaccines against infection and severe outcomes that explicitly handle the underreporting in vaccination.
Our study also has several potential limitations. First, effectiveness was investigated against documented infection in a cohort without previous infection, whereas the potential inclusion of patients with undocumented infections exists. Nevertheless, if this potential lack of data is evenly distributed across treatment groups, it could potentially attenuate the true vaccine effectiveness, thereby making our analysis more conservative in its estimations. Our inclusion of previous negative COVID-19 tests as a confounder aims to balance the probability of testing between treatment groups, which could partially adjust for this factor. Moreover, the increasing availability of at-home rapid antigen testing kits over time could have further reduced the testing frequency captured by the EHR. However, persons with severe illness who test positive through home kits typically seek medical care and report their results to hospitals. This would lead to a better capture of severe infection in our database and more reliable vaccine effectiveness estimates. Baseline confounders were balanced between vaccinated and unvaccinated groups, which should adjust for between-cohort differences in exposure risks and risks for severe infections. Second, as in any observational study, assignment to the vaccine group was nonrandom and the validity of the results could be affected by unmeasured confounders. To evaluate the effect of unmeasured confounders and residual bias, we conducted negative control experiments that quantified the robustness of our results.
Third, in this study, patients who had received vaccinations before the start of the study period were excluded. Due to missing vaccine records, some patients who had previously been vaccinated may have still entered the cohort, particularly in the unvaccinated group. However, the CER method used in this study adjusted for potential bias resulting from unrecorded vaccinated patients, which could also reduce the bias resulting from this issue. Finally, in the Omicron study involving adolescents, the cohort included adolescents who had their first vaccine after 1 January 2022. Because the use of BNT162b2 vaccines was authorized in adolescents aged 12 to 15 years on 10 May 2021, this cohort may represent a population with late vaccines, which reduces the generalizability of the findings.
Although this study provides evidence of a slight waning of vaccine effectiveness 4 months after the first dose against Omicron infection and the effectiveness is stabilized after 4 months, waning can be affected by vaccines during the follow-up period and other factors. Patients who received boosters during the follow-up period were not excluded from the study. A sensitivity analysis evaluating the durability of 2-dose vaccine effectiveness considering the third dose as censoring did not suggest a statistically significant different conclusion. A future study is warranted to investigate the effect of booster vaccination among children and adolescents. Furthermore, despite the recognized risk for myocarditis associated with COVID-19 vaccines in young men and teen boys, the study shows a lower relative risk for myocarditis, pericarditis, or MIS in vaccinated groups, which may be partially explained by reduced likelihood of infection during the study period (34).
In conclusion, this study involving national pediatric cohorts in the United States estimates moderate effectiveness of the BNT162b2 vaccine for preventing infection and severe diseases of the SARS-CoV-2 Omicron variant and high effectiveness against the Delta variant. The study shows a lower risk for cardiac complications among children and adolescents who were vaccinated during the Omicron period and no significant difference between vaccinated and unvaccinated groups during the Delta period. Our assessment of vaccine effectiveness across diverse outcomes underscores the vaccine's pivotal role in reducing SARS-CoV-2 transmission, minimizing COVID-19–related sick leaves, and alleviating economic burdens during the pandemic. This study substantially contributes to our knowledge of the BNT162b2 vaccine in the U.S. pediatric population using a rigorously designed CER method accounting for the incomplete capture of vaccination status in EHR data in the United States.
Supplemental Material
Supplementary Material
- Download
- 9.30 MB
Study Protocol
- Download
- 657.97 KB
References
1.
Flaxman S, Whittaker C, Semenova E, et al. Assessment of COVID-19 as the underlying cause of death among children and young people aged 0 to 19 years in the US. JAMA Netw Open. 2023;6:e2253590. [PMID: 36716029] doi: 10.1001/jamanetworkopen.2022.53590
2.
Frenck RW Jr, Klein NP, Kitchin N, et al; C4591001 Clinical Trial Group. Safety, immunogenicity, and efficacy of the BNT162b2 Covid-19 vaccine in adolescents. N Engl J Med. 2021;385:239-250. [PMID: 34043894] doi: 10.1056/NEJMoa2107456
3.
Walter EB, Talaat KR, Sabharwal C, et al; C4591007 Clinical Trial Group. Evaluation of the BNT162b2 Covid-19 vaccine in children 5 to 11 years of age. N Engl J Med. 2022;386:35-46. [PMID: 34752019] doi: 10.1056/NEJMoa2116298
4.
Chemaitelly H, AlMukdad S, Ayoub HH, et al. Covid-19 vaccine protection among children and adolescents in Qatar. N Engl J Med. 2022;387:1865-1876. [PMID: 36322837] doi: 10.1056/NEJMoa2210058
5.
Cohen-Stavi CJ, Magen O, Barda N, et al. BNT162b2 vaccine effectiveness against Omicron in children 5 to 11 years of age. N Engl J Med. 2022;387:227-236. [PMID: 35767475] doi: 10.1056/NEJMoa2205011
6.
Florentino PTV, Alves FJO, Cerqueira-Silva T, et al. Vaccine effectiveness of CoronaVac against COVID-19 among children in Brazil during the Omicron period. Nat Commun. 2022;13:4756. [PMID: 35963844] doi: 10.1038/s41467-022-32524-5
7.
Olson SM, Newhams MM, Halasa NB, et al; Overcoming Covid-19 Investigators. Effectiveness of BNT162b2 vaccine against critical Covid-19 in adolescents. N Engl J Med. 2022;386:713-723. [PMID: 35021004] doi: 10.1056/NEJMoa2117995
8.
Sacco C, Del Manso M, Mateo-Urdiales A, et al; Italian National COVID-19 Integrated Surveillance System and the Italian COVID-19 vaccines registry. Effectiveness of BNT162b2 vaccine against SARS-CoV-2 infection and severe COVID-19 in children aged 5-11 years in Italy: a retrospective analysis of January-April, 2022. Lancet. 2022;400:97-103. [PMID: 35780801] doi: 10.1016/S0140-6736(22)01185-0
9.
Khan FL, Nguyen JL, Singh TG, et al. Estimated BNT162b2 vaccine effectiveness against infection with Delta and Omicron variants among US children 5 to 11 years of age. JAMA Netw Open. 2022;5:e2246915. [PMID: 36515946] doi: 10.1001/jamanetworkopen.2022.46915
10.
PEDSnet. Accessed at https://pedsnet.org/ on 21 May 2023.
11.
Data_Models_Public/PEDSnet. Accessed at https://github.com/PEDSnet/Data_Models_Public/tree/master/PEDSnet on 21 May 2023.
12.
Hernán MA, Robins JM. Using big data to emulate a target trial when a randomized trial is not available. Am J Epidemiol. 2016;183:758-764. [PMID: 26994063] doi: 10.1093/aje/kwv254
13.
Suchard MA, Schuemie MJ, Krumholz HM, et al. Comprehensive comparative effectiveness and safety of first-line antihypertensive drug classes: a systematic, multinational, large-scale analysis. Lancet. 2019;394:1816-1826. [PMID: 31668726] doi: 10.1016/S0140-6736(19)32317-7
14.
Simon TD, Cawthon ML, Popalisky J, et al; Center of Excellence on Quality of Care Measures for Children with Complex Needs (COE4CCN). Development and validation of the Pediatric Medical Complexity Algorithm (PMCA) version 2.0. Hosp Pediatr. 2017;7:373-377. [PMID: 28634166] doi: 10.1542/hpeds.2016-0173
15.
Forrest CB, Burrows EK, Mejias A, et al. Severity of acute COVID-19 in children <18 years old March 2020 to December 2021. Pediatrics. 2022;149:e2021055765. [PMID: 35322270] doi: 10.1542/peds.2021-055765
16.
Berger JO, Liseo B, Wolpert RL, et al. Integrated likelihood methods for eliminating nuisance parameters. Stat Sci. 1999;14:1-28. doi: 10.1214/ss/1009211804
17.
Huang J, Duan R, Hubbard RA, et al. PIE: a prior knowledge guided integrated likelihood estimation method for bias reduction in association studies using electronic health records data. J Am Med Inform Assoc. 2018;25:345-352. [PMID: 29206922] doi: 10.1093/jamia/ocx137
18.
Dickerman BA, Gerlovin H, Madenci AL, et al. Comparative effectiveness of BNT162b2 and mRNA-1273 vaccines in U.S. veterans. N Engl J Med. 2022;386:105-115. [PMID: 34942066] doi: 10.1056/NEJMoa2115463
19.
Schuemie MJ, Hripcsak G, Ryan PB, et al. Empirical confidence interval calibration for population-level effect estimation studies in observational healthcare data. Proc Natl Acad Sci U S A. 2018;115:2571-2577. [PMID: 29531023] doi: 10.1073/pnas.1708282114
20.
Schuemie MJ, Ryan PB, DuMouchel W, et al. Interpreting observational studies: why empirical calibration is needed to correct p-values. Stat Med. 2014;33:209-218. [PMID: 23900808] doi: 10.1002/sim.5925
21.
Carlin BP, Louis TA. Bayesian Methods for Data Analysis. 3rd ed. CRC Pr; 2008.
22.
Rudan I, Millington T, Antal K, et al. BNT162b2 COVID-19 vaccination uptake, safety, effectiveness and waning in children and young people aged 12-17 years in Scotland. Lancet Reg Health Eur. 2022;23:100513. [PMID: 36189425] doi: 10.1016/j.lanepe.2022.100513
23.
Fleming-Dutra KE, Britton A, Shang N, et al. Association of prior BNT162b2 COVID-19 vaccination with symptomatic SARS-CoV-2 infection in children and adolescents during Omicron predominance. JAMA. 2022;327:2210-2219. [PMID: 35560036] doi: 10.1001/jama.2022.7493
24.
Lin D-Y, Xu Y, Gu Y, et al. Effects of COVID-19 vaccination and previous SARS-CoV-2 infection on Omicron infection and severe outcomes in children under 12 years of age in the USA: an observational cohort study. Lancet Infect Dis. 2023;23:1257-1265. [PMID: 37336222] doi: 10.1016/S1473-3099(23)00272-4
25.
Tan SHX, Cook AR, Heng D, et al. Effectiveness of BNT162b2 vaccine against Omicron in children 5 to 11 years of age. N Engl J Med. 2022;387:525-532. [PMID: 35857701] doi: 10.1056/NEJMoa2203209
26.
Block JP, Boehmer TK, Forrest CB, et al. Cardiac complications after SARS-CoV-2 infection and mRNA COVID-19 vaccination—PCORnet, United States, January 2021–January 2022. MMWR Morb Mortal Wkly Rep. 2022;71:517-523. [PMID: 35389977] doi: 10.15585/mmwr.mm7114e
27.
Patone M, Mei XW, Handunnetthi L, et al. Risks of myocarditis, pericarditis, and cardiac arrhythmias associated with COVID-19 vaccination or SARS-CoV-2 infection. Nat Med. 2022;28:410-422. [PMID: 34907393] doi: 10.1038/s41591-021-01630-0
28.
Barda N, Dagan N, Ben-Shlomo Y, et al. Safety of the BNT162b2 mRNA Covid-19 vaccine in a nationwide setting. N Engl J Med. 2021;385:1078-1090. [PMID: 34432976] doi: 10.1056/NEJMoa2110475
29.
Chin SE, Bhavsar SM, Corson A, et al. Cardiac complications associated with COVID-19, MIS-C, and mRNA COVID-19 vaccination. Pediatr Cardiol. 2022;43:483-488. [PMID: 35258639] doi: 10.1007/s00246-022-02851-x
30.
Boehmer TK, Kompaniyets L, Lavery AM, et al. Association between COVID-19 and myocarditis using hospital-based administrative data – United States, March 2020-January 2021. MMWR Morb Mortal Wkly Rep. 2021;70:1228-1232. [PMID: 34473684] doi: 10.15585/mmwr.mm7035e5
31.
Dufort EM, Koumans EH, Chow EJ, et al; New York State and Centers for Disease Control and Prevention Multisystem Inflammatory Syndrome in Children Investigation Team. Multisystem inflammatory syndrome in children in New York State. N Engl J Med. 2020;383:347-358. [PMID: 32598830] doi: 10.1056/NEJMoa2021756
32.
Belot A, Antona D, Renolleau S, et al. SARS-CoV-2-related paediatric inflammatory multisystem syndrome, an epidemiological study, France, 1 March to 17 May 2020. Euro Surveill. 2020;25:2001010. [PMID: 32524957] doi: 10.2807/1560-7917.ES.2020.25.22.2001010
33.
Schuemie MJ, Soledad Cepede M, Suchard MA, et al. How confident are we about observational findings in health care: a benchmark study. Harv Data Sci Rev. 2020;2. [PMID: 33367288] doi: 10.1162/99608f92.147cc28e
34.
Block JP, Boehmer TK, Forrest CB, et al. Cardiac complications after SARS-CoV-2 infection and mRNA COVID-19 vaccination - PCORnet, United States, January 2021-January 2022. MMWR Morb Mortal Wkly Rep. 2022;71:517-523. [PMID: 35389977] doi: 10.15585/mmwr.mm7114e1
Information & Authors
Information
Published In
Annals of Internal Medicine
Volume 177 • Number 2 • February 2024
Pages: 165 - 176
History
Published online: 9 January 2024
Published in issue: February 2024
Keywords
- Age groups
- Cardiology and cardiovascular diseases
- Cardiovascular diseases
- Children
- COVID-19
- Heart diseases
- Hospital medicine
- Pericardial diseases
- Pericarditis
- Population statistics
- Prevention, policy, and public health
- Preventive medicine
- Pulmonary diseases
- Respiratory infections
- Upper respiratory tract infections
- Vaccination and immunization
- Vaccines
Copyright
© 2024 American College of Physicians.
Authors
Metrics & Citations
Metrics
Citations
If you have the appropriate software installed, you can download article citation data to the citation manager of your choice. For an editable text file, please select Medlars format which will download as a .txt file. Simply select your manager software from the list below and click Download.
For more information or tips please see 'Downloading to a citation manager' in the Help menu.
Download article citation data for:
Real-World Effectiveness of BNT162b2 Against Infection and Severe Diseases in Children and Adolescents. Ann Intern Med.2024;177:165-176. [Epub 9 January 2024]. doi:10.7326/M23-1754
Real-World Effectiveness of BNT162b2 Against Infection and Severe Diseases in Children and Adolescents. Ann Intern Med.2024;177:165-176. [Epub 9 January 2024]. doi:10.7326/M23-1754
View More
Get Access
Login Options:
Purchase
You will be redirected to acponline.org to sign-in to Annals to complete your purchase.
Create your Free Account
You will be redirected to acponline.org to create an account that will provide access to Annals.
Unjustified assumptions about vaccination status
In this study, positive COVID vaccination status was determined from the EHR and is expected to be nearly 100% accurate. However, determination of negative COVID vaccination status was defined to be the absence of an EHR record of COVID vaccination. As such, the authors correctly observe that negative COVID vaccination status may be less than 100% accurate. False negative vaccination status occurs, for example, when a study subject receives COVID vaccination in a pharmacy not associated with one of the study institutions. The authors provide evidence that the effect of uncertainty of vaccination status on estimates of vaccine efficacy are limited provided that the "sensitivity" of the EHR to positive vaccination is 0.8 or greater for children and 0.7 - 0.9 for adolescents. However, the authors do not justify those ranges, only stating: "a range of possible sensitivities based on our prior study was prespecified for each study." No reference to that study is provided. These unjustified assumptions raise doubts about the study methodology and should limit confidence in the study conclusions.
Author Response to Yim
Note that some of the EHR data partners have augmented their data quality by including the vaccine records from a Health Information Exchange (HIE) program specifically focused on vaccine registries. The range of possible sensitivities was prespecified based on a pilot study, which compared the vaccination rates estimated from vaccine data obtained from health systems against the Centers for Disease Control and Prevention (CDC) reports. Specifically, a reference vaccination rate was calculated by weighting the CDC's county-level vaccination statistics by the patients’ residential addresses. This comparison between the health system-derived vaccination rates and the CDC's reference rates allowed us to predefine a range of possible values of sensitivity. The research paper detailing this pilot study is being finalized and will be submitted soon. For the pediatric population who receive their care during the pandemic from the participating institutions (all of which provided vaccination programs to the communities as well as the primary care physician units). A sensitivity of lower than 70% would be found uncommon.
In addition, to evaluate the robustness of findings from the statistical methods we used to account for the underreporting, we have conducted sensitivity analyses using alternative methods, including the naive method (without adjusting for underreporting) and using different ranges of possible sensitivity values. Please refer to Section 16 of the Supplement for details.
Disclosures:
Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M23-1754
Response to authors
One question that may arise is the extent to which the EHR data partners with the augmented data sets overlapped with the study of the accuracy of COVID vaccination status classification.