Natural Language Processing and Machine Learning for Detection of Respiratory Illness by Chest CT Imaging and Tracking of COVID-19 Pandemic in the United States

Introduction

The pandemic of COVID-19 (1) has spread quickly throughout the United States causing a significant disruption in health care (2,3) and society (4). Several countries around the world, including the United States, have been in shutdown for weeks to “flatten the curve” to mitigate COVID-19 disease spread and prevent local outbreaks that could threaten lives and overwhelm hospitals. Medical analytic tools that assess the early rise of possible infection and identify hot spots are important for public health planning and management. Inadequate availability of accurate testing, early identification, and case tracking were failure points in the initial U.S. (5) pandemic response, resulting in more than 1.5 million cases and over 90 000 deaths as of May 18, 2020, and progressing to more than 7.5 million cases and over 215 000 deaths as of October 3, 2020 (6). Reverse-transcription polymerase chain reaction (RT-PCR) nasal swab is the reference standard molecular test for COVID-19 diagnosis confirmation; however, an important issue with the RT-PCR test is the risk of false-negative or false-positive results (7). In particular, studies have shown that RT-PCR can lead to false-negative results due to insufficient collection of material or due to testing at a stage of the disease with lower viral shedding (8). There is increasing evidence supporting the importance of chest CT interpretation for patients with suspected COVID-19 (9–11) and its role in the assessment of false-negative RT-PCR results (8,12,13). Although chest CT imaging is not recommended for mass initial triage, proper use of chest CT can serve as an important evaluation component in patients presenting with moderate-to-severe symptoms and in several specific scenarios, such as defining the severity of disease for determining admission and appropriate level of care, providing an alternative differential diagnosis, or evaluating patients with worsening symptoms who may have treatable complications of progressive COVID-19 (eg, pulmonary embolism) (14,15). Typical chest CT findings have been described in patients with COVID-19 with high sensitivity (>90%) (13). Lung abnormalities during the early course of COVID-19 usually are peripheral multifocal ground-glass opacities affecting both lungs, and, as the disease progresses, air-space opacities and crazy paving (intralobular septal thickening) become more common CT findings (16). These typical chest CT imaging findings can be helpful in early diagnosis of suspected cases and in evaluating the severity and disease extent.

Progressive respiratory failure due to severe acute respiratory syndrome is the primary cause of death in the COVID-19 pandemic (17). Recently, work with artificial intelligence (AI) and machine learning (ML) demonstrates promising results in predicting COVID-19 spread and distinguishing it from other diseases such as pneumonia (18–20). Robust AI and ML strategies utilizing big data capable of extracting image findings from scans obtained throughout the United States with search engines relying on natural language processing (NLP) (21) are needed. The goal of our study was to determine if a ML-based NLP algorithm could perform syntactic analysis of chest CT imaging radiology reports to extract keywords for generating predictions and to compare such predictions with officially reported COVID-19 cases and deaths in the United States.

Materials and Methods

Using de-identified reports from our common imaging platform, which is interconnected with over 2100 facilities throughout all 50 U.S. states, we developed three NLP algorithms to track and display positive CT imaging features of respiratory illness and pulmonary findings that are typical of COVID-19. The NLP development was performed as part of an internal quality improvement project and to track and monitor the progression of respiratory illness by chest CT findings, specifically during a national pandemic. Institutional review board approval was waived, and patient consent was not required. This study was monitored and approved by our internal quality and safety committee. Our nationwide radiology practice provides interpretation of more than 20 000 examinations per day and more than 1000 chest CT studies per day using the same common imaging platform and dictation system.

To collect data used to train the NLP models, a list of keywords was selected to identify possibly relevant reports for each model. For the first NLP model, named Viral Pneumonia NLP, the keywords used were “ground-glass opacities,” “crazy paving,” “viral pneumonia,” “viral infection,” “covid,” “sars-cov,” and “coronavirus.” The second NLP model, named Imaging Findings NLP, used the keywords “bilateral ground-glass opacities,” “bilateral hazy opacities” and “crazy paving.” The third NLP model, named COVID NLP, used the keywords “viral pneumonia,” “atypical pneumonia,” “viral infection,” “covid,” and “coronavirus.” We selected the above three NLP models to compare a more general NLP algorithm including findings for viral pneumonia features (Viral Pneumonia NLP); one more specific NLP for typical imaging findings present in patients with COVID-19 (Imaging Findings NLP) and the last NLP to include specific words in the interpretation of free text reports indicating viral pneumonia and COVID-19 incidence (COVID NLP). All chest CT reports from January 1, 2020 to March 15, 2020 were screened for these keywords as a training dataset, and any report containing them was extracted. The Findings and Impressions sections of these clinical reports were split into sentences and preprocessed to remove nonalphanumeric characters. For each NLP model, these sentences were labeled as positive or negative for disease. Sentences were also labeled as negative if they contained any of the following keywords: tree-in-bud, air-space trapping, centrilobular, pleural effusion, atelectasis, and masses. Once sentences for a model were labeled, they underwent similar NLP methodology to prior published work from our group (22). The keyword extraction NLP uses an unsupervised ML approach (ie, the algorithm does not require training on a corpus nor any predefined rules, dictionary, or thesaurus). Instead, statistical features from the text itself are used and as such can be applied to large reports easily without retraining. The pipeline for extracting key phrases was to preprocess the radiologist report document to remove less informative common words and punctuation.

These three NLP models were then wired into our common imaging platform. A filter was first applied to screen for chest CT reports containing any keyword relevant to an NLP model; when a keyword was detected, the NLP model was applied to that report to determine whether sentence context was positive or negative for the given disease. Similar to training, the NLP models extract the Findings and Impressions report sections and preprocess the sentences prior to analysis. These NLP models were then applied to all clinical reports that passed through our system from January 1, 2020 to October 3, 2020, and the results were saved in our database.

All chest CT imaging studies were performed as part of clinical care in patients following typical indications from the American College of Radiology for the use of chest CT. Structured reporting is used in our common imaging platform. Radiologists received educational training regarding the typical imaging findings present in patients with suspected or confirmed COVID-19 infection. For data visualization and analysis, we used Microsoft Power BI, Power View (Microsoft, Redmond, Wash) and R (R Foundation). Power View engine utilized ML to synchronize the data and display the data in real time, and this software was used to create a temporal data display with a geomap of the United States (Movie). GitHub (subsidiary of Microsoft which provides hosting for software development and data repository) served as the data repository. Three authors (R.C.C., I.M., and T.L.) had full access to the data and performed the analysis.

To track the rate of reported COVID-19 cases and deaths in the United States over time, we used the same data repository that is being used by the COVID-19 Data Repository by the Center for Systems Science and Engineering (CSSE) at Johns Hopkins University (JHU), or “JHU CSSE COVID-19 Data,” which has been previously validated as a credible source to track new COVID-19 cases and deaths (23) and is available at the following website: https://github.com/CSSEGISandData/COVID-19.

Results

The three NLP algorithms were applied to 450 114 chest CT studies performed from January 1, 2020 to October 3, 2020. There were 107 120 positive cases (23.8%) flagged by the Viral Pneumonia NLP, 22 267 cases flagged by the Imaging Findings NLP (4.9%), and 21 202 cases flagged by the COVID NLP (4.7%). The correlation matrix is presented in the Table. We demonstrate that the Viral Pneumonia NLP, Imaging Findings NLP, and COVID NLP had a correlation (r) of 0.37, 0.81, and 0.91 with the official number of COVID-19 cases, respectively, with overall higher correlation for the more specific models (Viral Pneumonia NLP < Imaging Findings NLP < COVID NLP). The Imaging Findings NLP and COVID NLP had a correlation of 0.29 and 0.49 with the official number of COVID-19 deaths. There was no correlation between the Viral Pneumonia NLP and number of COVID-19 deaths. We can also observe that there was a very good correlation between the Imaging Findings and COVID NLP with correlation coefficient of 0.91 and a moderate correlation between Viral Pneumonia NLP and Imaging Findings NLP with correlation coefficient of 0.63.

Correlation Matrix Demonstrating Correlation among the Variables

Weekly Correlation

We selected the two best performing NLPs (Imaging Findings NLP and COVID NLP) to display the data week by week from the beginning of the year to compare the progression of the models with the number of official COVID-19 cases and deaths (Fig 1). The NLP models had a material increase in number of cases during the first wave starting in early March (week 11) and peaking in late March/early April (week 14). There was a strong correlation with the COVID NLP when compared with the number of new official COVID-19 cases on a weekly basis (r² = 0.91, P < .005). It is important to note that the early rise of cases detected by both NLP models occurred before the rise of new official COVID-19 cases during the first wave.

Figure 1:

Download as PowerPoint

State Correlation

There was an average of 540 chest CT studies analyzed in our platform per 1 million residents per state during the study period. The correlation coefficient was 0.88 for the COVID NLP and 0.89 for the Imaging Findings NLP when compared with new official COVID-19 cases on a state level, including all 50 U.S. states, Washington DC, and Puerto Rico (Fig 2). There was a strong correlation with the COVID NLP when compared with new COVID-19 cases by state (r² = 0.92, P < .005). The top states in number of cases identified by our more specific NLP model (COVID NLP) were Texas, California, Florida, New York, and Georgia, as displayed in Figure 2.

Figure 2:

Download as PowerPoint

Temporal Correlation

Finally, we tracked the progression of new COVID-19 cases in all 50 states using a real-time interactive U.S. map model displaying the temporal relationship between the COVID-19 pandemic and our NLP models. This interactive real-time temporal model was created using a ML algorithm with live input from our NLP models connected with our imaging platform and a direct feed from the JHU CSSE data repository (Fig 3 panel and Movie).

Figure 3:

Download as PowerPoint

Forecast and Future Prediction of New Cases

The first forecast model based on ML is presented in Figure 4, A, and can predict the next 10 days. As an example, as of November 20, 2020, the model was predicting 190 858 new COVID-19 cases on November 26, 2020 (95% CI: 167 197, 205 330). The second forecast model utilizes forecasting methodology to model time series data with complex seasonal patterns using exponential smoothing with no constraints to create detailed, long-term forecasts. The second model is presented in Figure 4, B, and can predict a longer period with appropriate confidence intervals. As an example, as of November 20, 2020, the model was predicting 166 412 new COVID-19 cases on December 19, 2020 (95% CI: 125 920, 206 903). The actual JHU COVID-19 number of cases on December 19 was 193 947, which is within the confidence interval.

Figure 4:

Download as PowerPoint

Discussion

The COVID-19 pandemic is challenging our society and economy in an unprecedented way. Overall, the U.S. response to contain COVID-19 may not have been as effective as other countries. This may have been due to insufficient or delayed testing and lack of alternative monitoring tools near the beginning of the pandemic (5). Early warning and detection may represent a critical opportunity for the United States to track the rate of respiratory illness and quickly institute policies to prevent or at least mitigate a future outbreak.

Strategies such as testing, contact tracing, and isolation of people who test positive will also be essential to successfully reopening state economies and keeping them open. Moreover, we believe it will also be fundamental to have tools to rapidly identify and track the geographically disproportionate emergence of respiratory illness and thereby identify hot spots of infection. Accelerated insights can be derived from aggregated data using ML and NLP (21,24) The present work uses big data and a ML-based NLP model to rapidly identify and monitor rates of respiratory illness as identified by chest CT imaging, based on key imaging findings previously reported in patients with COVID-19 infection. This work is distinctive because our common imaging platform is connected to over 2,100 facilities representing all 50 states. This provides an unparalleled opportunity to gather big data in real time that can be an accurate representation of regional and even local trends across the country. One of the main findings of our work is that the NLP models were able to detect imaging findings of respiratory illness before the rise of new official COVID-19 cases during the beginning of the pandemic. This is an interesting observation demonstrating the potential for this surveillance algorithm to flag respiratory illness as an early predictor of new COVID-19 cases and subsequent attendant mortality. Moreover, the NLP models had a strong correlation with the number of official new COVID-19 cases on a weekly level and on a state level.

The three NLP models used reflected a wide range of scenarios that could provide guidance to radiology departments in providing meaningful insights to hospitals and communities they serve. The first NLP model Viral Pneumonia NLP was a more general algorithm for viral infection including CT findings that were suggestive of viral infection, however, not necessarily viral infections that were specifically suggestive of COVID-19 and were quite prevalent even during the prepandemic season. The second NLP model Imaging Findings NLP focused on CT findings that were suggestive specifically of COVID-19. The third NLP model COVID NLP represented words of an interpretation of the radiologist as the findings being suggestive of viral infection or highly suggestive of COVID-19. It is interesting to note that the COVID NLP model was the best performing model with all correlations performed and highlights the ability of radiologists to summarize their findings in the impression and provide additional insights to clinical care. Other keywords, as presented by CO-RADS (25), should be considered for further correlation in future NLP studies, and this study serves as a baseline for such studies.

Real-time epidemiologic data are critical to managing different aspects of a pandemic. For instance, this data can help public health authorities to forecast demand/surge models, which may thereby allow public or private organizations to quickly reposition resources or reallocate personnel. These are corroborating data that should be used in combination with other indicators, such as the officially reported number of newly positive laboratory tests, disease-related hospitalizations, and disease-related deaths. Because our data are a marker of respiratory illness and of findings typical for viral pneumonia, it can serve as an additional indicator to predict the need for emergency medical services, hospital staff, hospital beds, personal protective equipment, and ventilator equipment, among others. Most cases of COVID-19 are mild (26); for this reason, early identification of a small excess number of severe and critical cases is crucial for planning hospital resource allocation. The extent and magnitude of imaging findings have been recently reported to correlate with worse outcomes and prognosis (15,27).

Our NLP model output may have included non-COVID viral pneumonia cases such as influenza, adenovirus, and other atypical pneumonias. However, we can clearly see the rapid rise of respiratory illness detected by our NLP models starting in early March when the COVID-19 pandemic started to progress rapidly in the United States. Therefore, while the absolute numbers could have included non-COVID-19 pneumonia cases, the relative increase and decrease in the total number of chest CT studies demonstrating respiratory illness and viral pneumonia may be very helpful to detect trends during an epidemic.

Limitations

As expected, we do not have access to all patients who have undergone chest CT imaging in the United States; however, our sample size is representative and provides an inclusive sample of all 50 states. Another limitation was the lack of a reference standard to confirm COVID-19 status in a patient undergoing chest CT, since a Health Level 7 interface to provide laboratory results for the patients was not available in all cases. The COVID-19 Data Repository by the CSSE at JHU is a credible source of data for new cases and deaths due to COVID-19, and the U.S. data are provided on a city level by the Centers for Disease Control and Prevention. Nevertheless, the utility of an early warning system using chest CT findings may in fact shine in the absence of laboratory data, as chest CT abnormalities can identify a regional spike in respiratory illness before a virus has even been isolated or, if already isolated, if viral testing is not yet widely available.

Conclusion

In conclusion, we developed a ML-based NLP algorithm to track respiratory illness imaging findings of viral pneumonia detected at chest CT and displayed as real-time data with strong correlation to the progression of the COVID-19 pandemic in the United States. This nationwide surveillance algorithm has the potential to help health care entities and public health authorities develop strategies against COVID-19 and other similar pandemics in the future. Future work will be required to further validate predictive forecast models based on NLP findings.