Assessing COVID-19 pneumonia—Clinical extension and risk with point-of-care ultrasound: A multicenter, prospective, observational study

1.1 Background

The main cause of morbidity and mortality in COVID-19 is viral pneumonia, which can progress to acute respiratory distress syndrome. Proper evaluation of pulmonary involvement is critical for appropriate triage, risk stratification, and efficient allocation of medical resources. This is especially important since new, more contagious genetic variants are emerging on multiple continents—some with the ability to generate more reinfections—which could put additional stress on already overwhelmed acute care pathways.¹ Mounting evidence suggests that imaging studies such as computed tomography (CT) are helpful in the diagnosis of COVID-19 pneumonia. Findings on CT may precede clinical symptoms, and the degree of pulmonary involvement can help predict patient outcome.^2-8 However, CT is expensive and cumbersome. CT scans can be difficult to perform on unstable patients, operation of CT equipment requires extra personnel, and unavailability and high costs can be an issue, even in high-income countries. In addition, the risk of COVID-19 transmission necessitates stringent desinfection protocols, and cleaning of the radiology suite may lead to increased delays between uses.

Point-of-care ultrasound (POCUS) of the lung might circumvent these issues. Lung ultrasound has diagnostic accuracy comparable to CT—and superior to chest radiography—in multiple aetiologies of respiratory failure, from pneumonia to acute respiratory distress syndrome.^9-15 Recent studies show that lung ultrasound also has better diagnostic accuracy than chest radiography in diagnosing COVID-19 pneumonia.^{18, 19} Chest radiography is a poor diagnostic test in COVID-19 as it may miss up to 40% of confirmed cases.²⁰ Meanwhile, diagnostic accuracy of lung ultrasound approaches that of CT for COVID-19 pneumonia.^{21, 22} Lung ultrasound can safely exclude clinically relevant COVID-19 pneumonia and may aid COVID-19 diagnosis in high-prevalence situations.²³ The advantages of lung ultrasound over CT include being fast to operate, simple to clean, low investment and operating costs, portable, and easily repeatable. With proper personal protective equipment, lung ultrasound can be safely performed at the bedside within 10 minutes. Because of its usefulness in diagnosis and follow-up, lung ultrasound has become common practice, and even standard practice, in acute and critical care.^{16, 17}

1.2 Goals of this investigation

The literature comparing the degree of pulmonary involvement between lung ultrasound and CT, along with literature detailing a lung ultrasound's ability to stratify patients and predict outcomes, is limited to only single-center and/or retrospective studies.^{16, 17, 24, 25} In a recent meta-analysis, the World Health Organization (WHO) recommended that the value of lung ultrasound regarding both clinical outcomes and duration of hospital stay should be investigated further because of inconclusive evidence.²⁶

1.3 Importance

To address this knowledge gap, we prospectively assessed the value of lung ultrasound during initial emergency department (ED) presentation in COVID-19 risk stratification and prognostication. In addition, we also evaluated the correlation between lung ultrasound and CT results in quantifying lung involvement in patients with COVID-19.

2.1 Study design

We performed a multicenter prospective, observational study. The study was registered with The Netherlands Trial Register and approved by the local medical ethical committees.

2.2 Setting

We conducted the study in the following 3 large university hospitals in The Netherlands: the Radboud University Medical Center in Nijmegen, the Vrije Universiteit Medical Center in Amsterdam, and the Academic Medical Center in Amsterdam. They have 1065, 733, and 1002 beds, respectively. Each hospital has ≈30,000 ED visits per year.

2.3 Participants

Patients were recruited between March 19, 2020, and May 4, 2020. Inclusion criteria were presentation at the ED for acute internal medicine, presence of a certified sonographer, confirmed COVID-19, and lung ultrasound and chest CT performed within 24 hours of presentation. Exclusion criteria were either no verbal consent and/or uninterpretable CT or lung ultrasound (Figure 1).

Per hospital procedure, all patients received a standard medical work-up (history, physical examination, common observations, and routine laboratory tests) and a SARS-CoV-2 polymerase chain reaction (PCR) test. Clinical criteria for ward admission were oxygen saturation <94% (need for conventional supplemental oxygen) and/or respiratory rate >20/minute. ICU admission criteria were either deterioration despite conventional respiratory support or need for mechanical ventilation.

2.4 Lung ultrasound

Lung ultrasound was performed or supervised by acute internal medicine physicians who are both certified in POCUS and had an Entrustable Professional Activity level of at least 4. The Entrustable Professional Activity concept has competency-based education targets to guarantee that all learners have a sufficient level of proficiency when they reach the required Entrustable Professional Activity level. Increasing levels of entrustment range from level 1 (not trusted to perform POCUS even under direct supervision) to level 4 (entrusted to use POCUS independently) and level 5 (engagement in POCUS education and research).²⁷ We predominantly used handheld systems, with settings amenable to the detection of B-line artefacts.²⁸ The scanning physician was blinded for the PCR and CT results, but not for the clinical picture.

The lung ultrasound protocol consisted of a structured assessment of 6 zones of each hemithorax (Figure 2). Each zone was scored according to the same lung ultrasound score (LUS) classification system used in intensive care, acute respiratory distress syndrome, and recent COVID-19 literature.^{15, 16, 21, 24, 29, 30} This system scores zones from 0 (well aerated) to 3 (consolidated) (Figure S1). The total LUS sums the scores from all 12 zones, creating a final score range from 0 (all regions are well aerated) to 36 (all regions are consolidated).

2.5 Computed tomography

Local radiologists assessed the chest CTs using the COVID-19 Reporting and Data System (CO-RADS).^{2, 31-33} The CO-RADS uses a scale from 1 (very low) to 5 (very high) to indicate the likelihood of COVID-19 pneumonia. To quantify pulmonary involvement, every CT is graded according to the CT severity score (CTSS), when COVID-19 pneumonia is suspected (ie, CO-RADS 3 and higher).³⁴ The CTSS is a visual assessment of the percentage of disease involvement in each lobe (Table S1). The total CTSS is the sum of the individual lobar scores and can range from 0 (no involvement) to 25 (maximum involvement). The radiologists were blinded for lung ultrasound and PCR results, but not for clinical information.

2.6 COVID-19 diagnosis

The diagnosis of COVID-19 was established by a positive PCR test result (from a nasopharyngeal, oropharyngeal, sputum, or broncho-alveolar lavage sample) or by a clinician impression after alternative diagnoses was excluded. In patients with high clinical suspicion but negative PCR, a multidisciplinary team of hospital clinicians determined the presence of COVID-19 on the basis of clinical, laboratory, microbiological, and/or CT data and only after excluding alternative diagnoses. Use of clinician impression as a reference is in line with WHO recommendations because of the suboptimal sensitivity of the PCR test.²⁶ The multidisciplinary team is composed of specialists in infectious disease, respiratory disease, and microbiology and discusses all admitted (suspected) COVID-19 cases in a daily plenary meeting. The team did not use lung ultrasound in the determination of COVID-19 presence.

2.7 Outcomes

The primary outcome was poor COVID-19 outcome, defined as the composite endpoint of 30-day all-cause mortality or ICU admission. Secondary outcomes included hospital length of stay (days) and disease severity groups (ED discharge, hospital ward admission, ICU admission).

2.8 Statistical analysis

We used Kaplan-Meier curves and Cox proportional hazards regression to assess the relationship between the LUS and time to poor outcome (a composite endpoint of 30-day all-cause mortality or ICU admission) and admission duration. We dichotomized the LUS using the median. We tested the proportional hazards assumption by checking that the Kaplan-Meier curves do not cross and the log minus log curves run parallel. We assessed linearity for continuous variables by dividing them in tertiles and checking whether β coefficients systematically increase or decrease with increasing categories.

We reported hazard ratios (HRs; adjusted for any confounders if applicable) as effect size for the dichotomized LUS used in the Kaplan-Meier curves as well as for each per-point increase in the LUS separately. We considered the following variables as potential confounders for a relation between the LUS and outcome: duration of symptoms, age, sex, and comorbidities (chronic obstructive pulmonary disease, cardiovascular disease, hypertension, and diabetes mellitus).² We examined confounders separately to keep within the rule of 5–10 events per variable. We used a 10% change in the LUS β coefficient as confounder threshold.³⁵ We estimated a multivariate model, when allowed by the events-per-variable rule, in which we adjusted for all confounders identified.

We compared differences between the following 3 disease severity grades: (1) mild, no admission/ED discharge; (2) moderate, hospital/ward admission; and (3) severe, ICU admission. We also compared means and proportions between these groups by 1-way analysis of variance (ANOVA), chi-square test, Fisher exact test, or Kruskal-Wallis test as appropriate. We used the Tukey honestly significant difference (HSD) post hoc test to compare pairs of severity grades.

We determined the ability of the LUS to discriminate between disease severity by comparing receiver operating characteristic (ROC) curves. We determined the optimal cutoff with the Youden index. We also reported sensitivity, specificity, positive likelihood ratio, negative likelihood ratio, and diagnostic odds ratio.^36-38 A diagnostic odds ratio of >10 is considered good.³⁹

We also used a “gray zone” approach.⁴⁰ This method allows the binary constraint of a “black or white” decision to be avoided, which is often inappropriate for clinical or screening practice. Instead, our approach constructs a 3-zone division for quantitative tests and purposefully includes a gray zone of inconclusive test results. In this case, we used a negative likelihood ratio of 0.1 and 0.2 and a positive likelihood ratio of 10 and 5 for the cutoffs of the gray zone. At these likelihood ratios, the approximate posttest change in probability is ∼45% (large change) or ∼30% (moderate change).⁴¹

We employed multiple imputation by chained equations with predictive mean matching to account for any missing data, assuming a missing-at-random pattern. We pooled effect estimates from the imputation data sets using Rubin's rules. We generated m = 10 imputed data sets, because of ≈5%–10% missing data.⁴² We used all patient characteristics, laboratory values, and radiological information mentioned in Table 1 and Table S2 with ≤30% missing data for imputation. We compared descriptives (eg, means and standard deviations [SDs]) of the imputation model with the original data to make sure all imputed values were plausible. We conducted a sensitivity analysis by also performing a complete case analysis.³⁵

We quantified the correlation between the LUS and CTSS with the Pearson correlation coefficient. We assessed interobserver agreement for the LUS by means of the intraclass correlation coefficient (ICC) with a 2-way random effects model. A total of 3 sonographers with different levels of experience (Entrustable Professional Activity levels 3, 4, and 5) measured the LUS in a random sample of 20 patients. We used a 2-sided significance level of 5% for all analyses. We performed all analyses as well as the multiple imputation in SPSS version 26.0.

3.1 Associations between the LUS and patient outcomes

There were 17 poor outcome events (14.9%) within 30 days. Time to poor outcome was shorter in the group with an LUS ≥12 compared with the group with an LUS <12. None of the potential confounders altered the association between the LUS and poor outcome. We found an HR of 5.59 (95% confidence interval [CI], 1.26–24.80; P = 0.023) in the imputed data set, with a comparable result in the complete case analysis (Table 2, Figure 3).

Hospital length of stay was longer among patients with an LUS ≥12 (Table 3, Figure 4). We found a crude HR of 1.98 (95% CI, 1.34–2.93; P = 0.001). Correcting for confounders led to an increase in the adjusted HR of 2.24 (95% CI, 1.47–3.40; P < 0.001). Analyses on the imputed data set and the complete analysis yielded similar results.

3.2 Differences between disease severity groups

There was a statistically significant difference in mean LUS between the disease severity grades (mild, no admission; moderate, ward admission; and severe, ICU admission) as determined by 1-way ANOVA (P < 0.001) (see end of Table 1). The mean LUS data were 6.3 (SD, 4.4), 13.1 (SD, 6.4), and 18.0 (SD, 5.0), respectively. Tukey HSD post hoc tests for pairwise comparisons revealed significant differences in mean LUS for all pairs of groups.

3.3 Discriminative ability

The AUC of the ROC curve that depicts the ability of the LUS to discriminate between hospital admission (ward and ICU combined) and no admission was 0.83 (95% CI, 0.75–0.91), which was almost identical to the CTSS (Figure 5, Figure S1). The optimal cutoff was 12, with a sensitivity of 70.2%, a specificity of 91.3%, and a diagnostic odds ratio of 24.8.

Gray zone analysis with a high LR cutoff (ie, negative likelihood ratio of 0.1 and positive likelihood ratio of 10) showed that patients with an LUS ≤1 (ie, LUS showing almost no signs of pulmonary involvement) could be discharged from the ED. We found that patients with an LUS ≥15 almost certainly required admission (Figure 6A). The gray zone (inconclusive test zone) contained 55.3% of patients. If we reduce the cutoff to a negative likelihood ratio of 0.2 and a positive likelihood ratio of 5, the number of patients within the gray zone decreases to 30.7% (Figure 6B). Patients with an LUS ≥11 had a posttest increase in admission probability of 30%. A total of 5 patients were admitted for reasons not related to the COVID-19 diagnosis, including only 1 patient without any abnormalities on the LUS and 4 patients without any CT abnormalities. In fact, all patients with an LUS <3 did not require any supplemental oxygen.

3.4 Correlation between the LUS and CTSS

We found a strong positive association between the LUS and CTSS, with a Pearson correlation coefficient of 0.60 (95% CI, 0.48–0.71). The interrater agreement for the LUS was excellent, with an ICC of 0.88 (95% CI, 0.77–0.95).

3.5 Limitations/bias

Our study has several limitations. First, although we included consecutive patients when a certified sonographer was present, we could not include every patient who tested positive for COVID-19. However, we tried to enroll every eligible patient when a certified sonographer was present, therefore minimizing any possible selection bias. We therefore feel that these omissions have not influenced our results. We look forward to seeing these results corroborated in different settings. Second, the wide CIs of the HRs in the poor outcome analysis are an indication of the imprecision of the estimate. This is most likely attributed to the relatively small sample size. Our results should, therefore, be confirmed in a larger study. Third, we also included patients who were diagnosed on the basis of a multidisciplinary team decision. However, the results do not differ between the PCR-positive and the combined PCR/multidisciplinary team–positive groups. Fourth, we tried to minimize incorporation bias by blinding the radiologists and scanning physicians to each other's results. However, they were aware of the patients’ clinical parameters. Although this may have caused an upward adjustment of the LUS and CTSS in patients who were sicker and vice versa in patients who were less sick, we would have expected this to apply to CT and the LUS to the same degree. Moreover, blinding them entirely from the patient's clinical information was not desirable as this does not reflect daily practice.