Quantitative Assessment of Chest CT Patterns in COVID-19 and Bacterial Pneumonia Patients: a Deep Learning Perspective

Study design and subject

We performed a retrospective cohort study of CT scans of 73 patients with COVID-19 infection obtained between February 2020 to March 2020, and 97 patients with bacterial pneumonia between March 2012 to February 2014 at Yeungnam University Medical Center, in Daegu, Korea. During the study period, all consecutive adult patients (age > 18 years) with SARS-CoV-2 infection admitted to the hospital were eligible for inclusion. SARS-CoV-2 infection was confirmed by real-time RT-PCR assay of nasal and pharyngeal swab samples. Severity was defined as a composite outcome of acute respiratory distress syndrome (ARDS), intensive care unit admission, or death. ARDS was diagnosed according to the Berlin definition.12 The National Early Warning Score (NEWS) is an early warning score, and is composed of seven parameters: pulse oximetry, oxygen, pulse rate, systolic blood pressure, respiration rate, temperature, and central nervous system status. NEWS showed excellent predictive value in predicting critical clinical outcomes of COVID-19. NEWS is stratified into three categories: low risk (0–4), medium risk (5–6), and high risk (≥ 7).13, 14 Fig. 1 shows the flow chart of data collection, exclusion and splitting ratios applied for training and evaluation.

Imaging pattern analysis using deep learning

To analyze the COVID-19 manifestations using deep learning, the proposed framework consists of 3 key modules: 1) lung and lesion segmentation, 2) deep feature extraction, and 3) K-means clustering modules, respectively. Fig. 2A shows a diagram of the framework.

A Mask-cascade-RCNN-ResNeSt-200 with deformable convolution neural network (DCN) architecture was employed in module (a) to extract the lung and lesion regions in the chest CT scans.15, 16 The lung segmentation model was trained and evaluated on a total 51,978 manually segmented slices (train: 50,756, test: 1,222) from two public datasets (NSCLC, 20 cases).17, 18 For the lesion segmentation task, 6,971 manually segmented slices (train: 5,854, test: 1,117) from three publicly available datasets were used (20 cases, MosMed, MSD).18, 19, 20 Then, for the data from Yeungnam University Medical Center, lesions were extracted using the trained models. The patches smaller than 13 mm were not used to avoid misclassification caused by wrong segmentation or noise such as motion artifacts, as this may have a negative effect on subsequent analysis.

The deep neural network in module (b) employed a ResNet50 model trained to differentiate lesion patches of COVID-19 from those of bacterial pneumonia patients.21 The model took a lesion cropped patch as input and returned a label prediction. A total of 12,235 lesion patches (train: 6,181, test: 6,054) from 170 patients were employed. A 2048-dimensional feature vector extracted from the intermediate layer of the ResNet50 model used in the clustering phase, i.e., module (c).

In module (c), the K-means algorithm was applied to cluster the lesion features into 20 groups.22 To profile the typical or relatively atypical imaging features of COVID-19, a total of 12,235 lesion patches from 170 patients were represented in two-dimensional space via a t-distributed Stochastic Neighbor Embedding based reduction of the 2,048-dimensional feature vectors.23 The lesion images in each cluster initially grouped by K-means were later manually described by one radiologist (Jongsoo Park) using imaging terms, wherein three pulmonologists (Kyung Soo Hong, Jong Geol Jang, and June Hong Ahn) evaluated the descriptions and reached a consensus.

Verification of selected imaging patterns

Patient-level diagnosis can be achieved by aggregating the lesion-level predictions using majority voting. However, in this way, it is difficult to analyze what patterns the patient has and which combination of patterns are highly correlated with the disease or severity. To further quantify the diagnostic performance of the extracted imaging patterns, we constructed a histogram of lesion types using the clusters obtained via module (c) for each CT. Fig. 2B shows a diagram of the evaluation framework. In addition, the constructed histograms were analyzed using an independent two-sample t-test to evaluate the significance of each cluster regarding: 1) COVID-19 and bacterial pneumonia patients, 2) severe and non-severe cases. A two-tailed P < 0.05 was taken to indicate statistical significance. Furthermore, we compared the mean histograms to confirm cluster relevance using the mean of each histogram in each group; i.e., COVID-19, bacterial pneumonia, severe, and non-severe. The cluster which showed high statistical significance (P < 0.001) showed significant value differences when performing mean histogram comparison.

Furthermore, we compared the classification accuracy between majority voting of the lesion-level predictions and Support Vector Machine (SVM) based inference which was trained on a combination of the proposed histogram with radiomics features. A total of 107 features were extracted using radiomics from each CT volume, including first-order statistics, shape-based features, etc. Finally, 20 features from our proposed histogram and 107 features from radiomics features were combined. We compared the performance with/without the combination of the radiomic features to verify effectiveness.

Statistical analysis

All statistical analyses were performed using Scipy (1.5.0, https://www.scipy.org/). The independent two-sample t-test was used to analyze the differences between groups, and Spearman's correlation test was used to analyze the correlation between laboratory data, clinical parameters and values of a cluster. The cluster which showed high statistical significance (P < 0.001) lead to significant value differences when performing mean histogram comparison. In all other analyses, P value < 0.05 was considered to indicate statistical significance.

Ethics statement

This study was conducted in accordance with the tenets of the Declaration of Helsinki and was reviewed and approved by the Institutional Review Board of Yeungnam University Hospital (YUH IRB 2020-05-030). The requirement for informed consent was waived due to the retrospective study design.

Demographic and clinical characteristics

Hospitalized patients with confirmed COVID-19 (73 patients) and bacterial pneumonia (97 patients) were included in this study (Fig. 1). Baseline characteristics of all patients are summarized in Table 1. The patients in the COVID-19 group were older than the patients in the bacterial pneumonia group (58.70 ± 16.49 vs. 39.86 ± 8.69, P < 0.001). 36 patients (49.3%) of COVID-19 and 55 patients (56.7%) of bacterial pneumonia were male. Body temperature (37.21 ± 0.67 vs. 37.76 ± 0.92, P < 0.001), and heart rate (86.70 ± 13.94 vs. 94.37 ± 16.91, P = 0.02) were significantly lower in patients with COVID-19. Systolic blood pressure (128.55 ± 19.44 vs. 115.71 ± 19.42, P < 0.001), and diastolic blood pressure (80.74 ± 12.15 vs. 71.42 ± 13.52, P < 0.001) were significantly higher for COVID-19 patients. CURB-65 was not significantly different in patients with COVID-19 and bacterial pneumonia (0.69 ± 0.87 vs 0.57 ± 0.68, P = 0.314). The mean time from the onset of symptoms to CT scan was 7.46 ± 4.04 days.

Deep chest CT manifestations

We summarize the deep chest CT manifestations observed in Table 2 and Supplementary Table 1, with lesion patches visualized in Fig. 3, respectively. Typical COVID-19 CT manifestations such as GGOs with interlobular septal thickening in the peripheral lungs were observed in clusters 4 and 17 (Fig. 4A). On the other hand, manifestations typical to bacterial pneumonia such as crazy-paving appearance in the posterior lungs were observed in clusters 5 and 10, as presented in Fig. 4B. These typical clusters showed P values less than 0.001.

Notably, 15 out of the 20 clusters showed key characteristics for the discrimination of COVID-19 from bacterial pneumonia with P values less than 0.001. Among 15 clusters, three clusters with P values less than 0.001 were shown to be key for severity classification. Moreover, two clusters could classify both COVID-19 and severe patients, with one of the clusters (#8) showing diffuse GGOs in the central and peripheral lungs which represents a typical severe COVID-19 CT manifestation. Fig. 4C presents the severe COVID-19 patient's CT manifestations.

In principle, the observed relative differences between lesion clusters are key to better understand the entire lesion distribution. As shown in Fig. 4D, extensive consolidation with air-bronchogram was observed in severe bacterial pneumonia patients. Compared with the severe COVID-19 patients who mainly showed diffuse GGOs, we could easily distinguish the difference in patterns observed in the later stages of disease onset.

To confirm cluster relevance, we compared the mean histograms of different tasks as presented in Fig. 5. The clusters which showed high significance (P < 0.001) in Table 2 also showed significantly different values in mean histogram comparison. Notably, the mean histogram of COVID-19 can be discriminated from bacterial pneumonia by comparing the relatively increased values in cluster 1, 4, 7, 12, and 17. On the other hand, the mean histogram of bacterial pneumonia can be discriminated from COVID-19 by comparing the significantly increased values of clusters 3, 6, 9, 11, 13, 14, and 18. In addition, the mean histogram of the severe COVID-19 patients showed consistently increased values in cluster 8 compared to the mean histogram of the non-severe COVID-19 patients. This indicates that the diagnosis is highly relevant to the presence of a lesion in a certain cluster, and the patient with a lesion in cluster 8 can be considered as a more severe COVID-19 patient than patients with lesions in other clusters. In summary, the cluster interpreted by the radiologist as being highly relevant to typical COVID-19 and bacterial pneumonia patterns showed high correlation with P value significance in statistical analyses, with notable differences observed in mean histogram comparison.

Disease and severity classification

In Table 3, we present disease classification accuracy across several metrics under different settings. First, the baseline method based on majority voting of predictions achieved 87.5% accuracy for COVID-19 patient classification. For the SVM classifier, we consider two scenarios, (i) with histogram features only or (ii) with radiomic features only. Notably, the classifier trained under the first setting achieved 88.7% accuracy; a minor improvement over the baseline, whereas in setting (ii)—81.25% was reported, a considerable decrease from the other models. However, when the SVM classifier was trained with both radiomics and histogram features, accuracy significantly improved to 91.2% from 87.5%. This further shows that the features learned by deep learning are highly robust for accurate patient diagnosis. In addition, the constructed histograms can accurately express the correlation of lesion features observed in each patient and/or can represent some combination of patterns that could lead to a severe outcome of a disease diagnosis. In comparison to the naïve aggregation method baseline i.e., majority voting, often highly limited in patient-level representations; the proposed deep feature representation highlights several key advantages.

Table 4 shows the severity classification accuracy. In terms of accuracy alone, SVM classifiers that only use either histogram or radiomics only achieved the same performance, i.e., 82.5% for both, with key differences noted in the sensitivity and specificity of the models. On the other hand, when the features were combined considerable improvement (+10%) was noted i.e., 95% accuracy, surpassing previous models. This may be attributed to the fact that the histogram is not able to represent the absolute size difference of lesions, thus the combination mitigates the issue. Rather than using a single feature alone, the combination of features proved to be invaluable for diagnosis.

Lung and lesion segmentation

The lung and lesion segmentation model achieved a dice coefficient score of 97.18%, and 78.06%, respectively. Fig. 6 shows the lesion segmentation results around the average dice coefficient score of 78.06%, which helps to qualitatively understand the accuracy of lesion segmentation.

Correlation between laboratory data, clinical and CT quantitative parameters

The correlation between laboratory data, clinical parameters and CT quantitative parameters are shown in Table 5. The values of cluster 8 were strongly correlated with NEWS (P < 0.001), and positively correlated with neutrophil percentage, procalcitonin, ARDS, and extracorporeal membrane oxygenation (ECMO) (P < 0.05). On the other hand, a negative correlation with lymphocyte percentage and lymphocyte count (P < 0.05) was noted. Comparisons between values of cluster 8 versus NEWS (Fig. 7A) and disease severity (Fig. 7B) in COVID-19 patients are shown. Notably, values were significantly higher among NEWS ≥ 7 patients than in NEWS 0–4 (P < 0.001), with severe COVID-19 patients also showing higher values than non-severe COVID-19 patients (P < 0.05), respectively.