Introduction

The virus SARS-CoV-2, responsible for causing the infectious disease known as COVID-19, has spread throughout the world causing the World Health Organization (WHO) to declare it as a pandemic on March 2020 [1, 2]. The combination of various diagnostic tests is the greatest option to maximize sensitivity and specificity for the diagnosis of COVID-19; however, a positive real-time reverse transcription polymerase chain reaction (RT-PCR) test remains the gold standard [3, 4]. Nevertheless, studies have suggested the potential of chest computed tomography (CT) to detect COVID-19 pneumonia imaging findings prior to RT-PCR positive testing [5], and similarly, to have comparable sensitivity to RT-PCR assays in diagnosing COVID-19 in epidemic areas with high pre-test probability [6]. However, common and characteristic chest CT imaging findings of COVID-19 patients can be similar to those caused by several other infectious and noninfectious entities [7, 8]. As a result, given the non-specific appearances, several radiological societies do not currently endorse utilizing chest CT for screening purposes or primary diagnosis of COVID-19. On the contrary, it is advised that chest CTs are retained for patients who are hospitalized and/or symptomatic with certain clinical indications, as well as for patients with suspicion of complicated COVID-19 pneumonia [9,10,11].

Under the prevailing circumstances, chest CTs may often be performed in the management of COVID-19 suspected and/or confirmed cases and be used as a complementary diagnostic tool. Initial chest CT may often be included in the workflow of some centers primarily in order to speed up patient triage [12]. This fact has created an increased need for an effective and standardized quantification of the disease extent as perceived from the chest CTs being performed. In order to deal with the burden and increased requirements that this pandemic has caused, since March 2020, we have been evaluating COVID-19 chest CTs based on an already published simplistic and easy-to-use CT percentage scaling system, which we named “Co.V.A.Sc.” (Covid Visual Assessment Scale) [13]. This quantitative tool is being employed in our COVID-19 reference hospital in an attempt to rapidly and efficiently stratify the imaging extent of COVID-19 disease while effectively shortening CT-result communications with clinicians. However, up until now, we had not finalized a clinical correlation of this scale.

The purpose of this article is to evaluate the interobserver and intraobserver variability of the formerly developed Covid Visual Assessment Scale (“Co.V.A.Sc.”) for chest CT and to estimate its probability and prospect to predict clinical outcome.

Materials and methods

Chest CT “Co.V.A.Sc.” definitions

The “Co.V.A.Sc.” (Covid Visual Assessment Scale) [13] is a percentage scaling system that an observer can use when assessing a chest CT of an RT-PCR-confirmed COVID-19 patient in order to roughly estimate and quantify the total extent of pulmonary parenchyma affected by COVID-19-associated pulmonary opacities (opacities are defined according to the Fleischner Society Glossary of Terms for Thoracic Imaging, as areas appearing more opaque compared to those adjacently, due to attenuation of the x-ray beam [14]).

When using this simple scale, both lungs are evaluated as a whole and the proposed groupings of the percentages of pulmonary parenchyma affected by COVID-19 disease are as follows: 0% (“Co.V.A.Sc.” unit 0—no disease), 1–10% (“Co.V.A.Sc.” unit 1—mild disease) (Fig. 1), 11–25% (“Co.V.A.Sc.” unit 2—mild to moderate disease) (Fig. 2), 26–50% (“Co.V.A.Sc.” unit 3—moderate disease) (Fig. 3), 51–75% (“Co.V.A.Sc.” unit 4—severe disease) (Fig. 4), and > 75% (“Co.V.A.Sc.” unit 5—critical disease) (Fig. 5).

Fig. 1
figure 1

Covid Visual Assessment Scale (“Co.V.A.Sc.”)—unit 1: 1–10%. Axial chest CT images obtained in a RT-PCR confirmed COVID-19-positive patient demonstrate pulmonary infiltrates affecting approximately 1–10% of the total lung parenchyma

Full size image
Fig. 2
figure 2

Covid Visual Assessment Scale (“Co.V.A.Sc.”)—unit 2: 11–25%. Axial chest CT images obtained in a RT-PCR confirmed COVID-19-positive patient demonstrate pulmonary infiltrates affecting approximately 11–25% of the total lung parenchyma

Full size image
Fig. 3
figure 3

Covid Visual Assessment Scale (“Co.V.A.Sc.”)—unit 3: 26–50%. Axial chest CT images obtained in a RT-PCR confirmed COVID-19-positive patient demonstrate pulmonary infiltrates affecting approximately 26–50% of the total lung parenchyma

Full size image
Fig. 4
figure 4

Covid Visual Assessment Scale (“Co.V.A.Sc.”)—unit 4: 51–75%. Axial chest CT images obtained in a RT-PCR confirmed COVID-19-positive patient demonstrate pulmonary infiltrates affecting approximately 51–75% of the total lung parenchyma

Full size image
Fig. 5
figure 5

Covid Visual Assessment Scale (“Co.V.A.Sc.”)—unit 5: > 75%. Axial chest CT images obtained in a RT-PCR confirmed COVID-19-positive patient demonstrate pulmonary infiltrates affecting > 75% of the total lung parenchyma

Full size image

The use of the “Co.V.A.Sc.” is endorsed for RT-PCR-confirmed COVID-19-positive patients demonstrating typical and/or indeterminate appearances according to the RSNA Expert Consensus Statement on Reporting Chest CT Findings Related to COVID-19, as seen in Table 1 [8]. However, employing the “Co.V.A.Sc.” is not endorsed in cases where assessment of COVID-19-related opacities and their extent is not reliable, and those must be explicitly communicated to the attending physicians. Such cases include patients demonstrating atypical appearances (see Table 1) [8] (since in this case, an alternate or coexisting diagnosis could be present despite RT-PCR positive results), and patients with conditions substantially altering the expected normal pulmonary parenchyma volume (i.e., pulmonectomy, lobectomy, lobar atelectasis, extensive pleuritic fluid causing significant passive atelectasis, extensive emphysematous lesions) according to the observer’s assessment.

Table 1 Consensus RSNA classification system for chest CT imaging findings related to COVID-19 with four categories and suggested reporting language (reproduced with permission from: Simpson S, Kay F U, Abbara S, et al. Radiological Society of North America Expert Consensus Document on Reporting Chest CT Findings Related to COVID-19: endorsed by the Society of Thoracic Radiology, the American College of Radiology, and RSNA. Radiology: Cardiothoracic Imaging 2020. Published online March 25, 2020. https://doi.org/10.1148/ryct.2020200152. © Radiological Society of North America [8])
Full size table

The “Co.V.A.Sc.” provides quantitative information regarding imaging burden of COVID-19 disease. Hence, qualitative findings, such as the type of pulmonary infiltrates (ground glass opacities, crazy-paving pattern, consolidations) which are the most commonly encountered [7, 8], should also be included in the radiology report, as they may provide valuable insight concerning the stage of COVID-19 disease [15].

Ethics

The study was conducted according to the principles of the Declaration of Helsinki for clinical studies and Institutional Review Board permission was obtained. Due to its retrospective nature, obtaining written informed consent for this study (reference number ΑΚΤΙΝ, ΕΒΔ 234/19-4-2021) was waived by (UGHA).

Patient selection and data collection

Initially, the electronic database of the Department of Radiology was retrospectively searched for all RT-PCR-confirmed COVID-19 patients (> 18 years old) undergoing chest CT within 48 h of admission to our hospital, between 01/03/2021 and 17/03/2021. Subsequently, patient demographics and comorbidities, number of days with clinical symptoms prior to CT, PO2/FiO2 ratio upon admission (PO2: arterial PO2 as found in arterial blood gas (ABG), FiO2 fraction (percent) of inspired oxygen that the patient is receiving expressed as a decimal (i.e., 50% oxygen = FiO2 of 0.50)), ICU admission and intubation, clinical outcome, and blood test results were retrieved from the electronic medical database. Patients with chest CT findings obviating the use of the “Co.V.A.Sc.” (according to its definition above) and patients with insufficient or absent clinical information were excluded from the study.

Imaging analysis and evaluation

For the purposes of this study, CT images were independently evaluated by two radiologists (N.A.A. & O.M.Z.), with 5 and 4 years of experience in interpreting chest CTs respectively. The reviewing radiologists were blinded to each other’s results and blinded to patient’s clinical information and laboratory results.

At first, both observers estimated the overall extent of COVID-19 pulmonary opacities using the “Co.V.A.Sc.,” in order to assess the interobserver agreement. The time needed for CT assessment by each observer was also recorded.

Subsequently, in order to assess the intraobserver agreement, after a period of 10 days, 50 CTs from the initial assessment were randomly chosen and reassessed by both observers, while being blinded to their own initial “Co.V.A.Sc.” evaluations.

CT examination procedure

All CT examinations were performed on a Philips Brilliance™ 64 CT scanner, dedicated to COVID-19 confirmed and/or suspected cases, using automatic tube current modulation, a tube voltage of 120 kV (ranging from 100 to 140 kV), and a slice thickness of 1 mm with a high-resolution CT algorithm. Scanning was performed with the patient in the supine position, with end-inspiration breath hold being instructed in individuals who could tolerate it. Examined patients typically underwent a non-contrast chest CT scan; however, some cases necessitated iodinated contrast administration in order to exclude suspected pulmonary embolism (PE).

Statistical analysis

Categorical variables are reported as absolute frequencies/percentage and continuous variables are reported as mean ± standard deviation (SD). Normality was evaluated with the Kolmogorov–Smirnov goodness-of-fit test. Independent sample t-test was performed for parametric comparisons and the Mann–Whitney U test for non-parametric comparisons. The possible correlation between clinical and laboratory variables with ICU admission or death was evaluated using binary regression analysis. The variables analyzed in the multivariable model were pre-defined (based on medical relevance) and included age, gender, comorbidities, PO2/FiO2 ratio, symptom duration, laboratory values such as CRP, procalcitonin, ferritin, lymphocytes, white blood cells, D-dimers, and troponin value analysis. To assess the inter- and intraobserver agreement, we used the interclass correlation coefficient. All statistical analysis, including inter- and intraobserver variability for “Co.V.A.Sc,” was performed with the IBM SPSS statistical software (version 26.0; SPSS/PASW, IL, USA) using p < 0.05 as the level of statistical significance.

Results

“Co.V.A.Sc.” imaging analysis and evaluation

Initial patient selection

During the study period (before the occurrence of the omicron SARS-CoV-2 variant [16]), 336 consecutive patients with RT-PCR-confirmed COVID-19 disease underwent a baseline chest CT scan within 48 h of admission. For the purposes of this study, 9 patients were excluded from assessment based on the “Co.V.A.Sc.” according to its definition. Analytically, 3 patients were excluded on the basis of demonstrating atypical appearances on chest CT [8] (Table 1), 3 patients were excluded due to extensive-panlobular emphysematous changes, 1 patient was excluded due to the presence of extensive bilateral pleural effusions causing significant passive atelectasis, and 2 patients were excluded due to lobar atelectasis caused by a concomitant hilar mass, which omitted evaluation of opacities in the atelectatic parenchyma. The remaining 327 patients had their admission chest CT evaluated by the two observers, according to the “Co.V.A.Sc.”

Interobserver and intraobserver agreement

Excellent inter- and intra-reader variability were noted between the two participating independent radiologists, with interclass correlation coefficient of 0.944 (95%CI 0.929–0.956) and 0.979 (95%CI 0.963–0.988), respectively. As a result, evaluations provided by one of the two readers were randomly chosen and used for further analyses.

Mean time assessment

The mean time required by the two observers (N.A.A. & O.M.Z.) for the evaluation of a chest CT according to the “Co.V.A.Sc.” was 20.98 ± 10.07 s (range: 7–51 s) and 22.02 ± 8.62 s (range: 9–46 s), respectively.

“Co.V.A.Sc.” clinical correlation

Final patient selection

Following imaging analysis and evaluation, 54 patients were excluded due to inadequate or absent clinical information. Finally, 273 hospitalized patients were included in the study.

Findings and correlation

Patients’ demographical and laboratory data are analytically reported in Table 2.

Table 2 Patients’ demographic data, comorbidities, outcome, and laboratory examinations on admission (n = 273)
Full size table

Overall, in-hospital mortality was 10.98% (30/273 patients). Twenty-five patients (25/273; 9.15%) were admitted to ICU. Mortality rate in ICU patients was 68% (17/25 ICU COVID patients died). Mean hospital stay was 7.7 ± 3.6 days (range: 1–25 days).

Chest CT findings in most of the patients were quantified as “Co.V.A.Sc.” unit 3 (110/273 cases; 40.3%), followed by “Co.V.A.Sc.” unit 2 (72/273 cases; 26.4%), “Co.V.A.Sc.” unit 4 (43/273 cases; 15.7%), “Co.V.A.Sc.” unit 1 (33/273 cases; 12.1%), “Co.V.A.Sc.” unit 0 (8/273 cases; 2.9%), and “Co.V.A.Sc.” unit 5 (7/273 cases; 2.5%).

According to the binary logistic regression analysis, only “Co.V.A.Sc.” classification (Exp(B) 0.391, 95%CI 0.212–0.719; p = 0.025) and patient age (Exp(B) 0.947, 95%CI 0.902–0.993; p = 0.25) were correlated with ICU admission. Specifically, the probability of ICU admission increased by 1.47 times for each unit increase of “Co.V.A.Sc.” Additionally, only age (Exp(B) 1.111, p = 0.0001), “Co.V.A.Sc.” (Exp(B) 2.408; p = 0.002), and male gender (Exp(B) 3.213; p = 0.028) were correlated with in-hospital mortality. Therefore, the probability of death increased by 11.1 times for each unit increase of “Co.V.A.Sc.”

According to receiver-operating characteristic (ROC) analysis, “Co.V.A.Sc.” could predict the admission of a patient in ICU with a diagnostic accuracy of 70.6% (AUC 0.706; p = 0.001) and the optimal “Co.V.A.Sc.” cutoff value to predict ICU admission was “Co.V.A.Sc.” unit 3 (sensitivity 56.0% and specificity 84.3%) (Fig. 6a). Furthermore, “Co.V.A.Sc.” could predict in-hospital death with a diagnostic accuracy of 64.1% (AUC 0.641; p = 0.01), and the optimal “Co.V.A.Sc.” cutoff value to predict in-hospital death was “Co.V.A.Sc.” unit 4 (sensitivity 41.9% and specificity 93.6%) (Fig. 6b).

Fig. 6
figure 6

Receiver-operating characteristic (ROC) curve indicating the optimal cutoff value of the “Co.V.A.Sc.” to predict a ICU admission and b in-hospital death

Full size image

No significant correlation was demonstrated between “Co.V.A.Sc.” classification, patient comorbidities, number of days with clinical symptoms prior to CT, PO2/FiO2 ratio upon admission, and laboratory blood tests.

Discussion

The COVID-19 pandemic continues to pose a relentless threat to health care systems around the world with numerous diagnostic and therapeutic measures being exploited in the fight against the disease. Several radiological societies endorse retaining chest CTs for patients who are hospitalized and/or symptomatic with specific clinical indications, as well as for patients with suspicion of complicated COVID-19 disease [9,10,11]. In everyday clinical practice, chest CTs constitute an important complementary tool in suspected and/or confirmed cases [12], causing a significant upsurge in the workload of radiology departments. This fact has created an increased need for rapid evaluation of the chest CTs being performed in a way that will additionally allow shortening CT-result communications with clinicians. For this purpose, since March 2020, we have been evaluating chest CTs obtained from RT-PCR-confirmed COVID-19 patients being imaged upon presentation/admission to our hospital, based on a simplistic and easy-to-use CT percentage scaling system, which we named “Co.V.A.Sc.” (Covid Visual Assessment Scale) [13], as previously described. In the current study, our key findings were that for each “Co.V.A.Sc.” unit increase, the probability of ICU admission increased by 1.47 times while the probability of death increased by 11.1 times. Specifically, we found that the optimal “Co.V.A.Sc.” cutoff value to predict ICU admission was “Co.V.A.Sc.” unit 3 (26–50% of overall lung parenchyma affected) and the optimal “Co.V.A.Sc.” cutoff value to predict in-hospital death was “Co.V.A.Sc.” unit 4 (51–75% of overall lung parenchyma affected).

The above findings are analogous to the available literature which indicates that CT extent of disease is associated with prognosis and can predict poor outcome [17]. More specifically, in a study conducted by Ahlstrand et al. [18], the authors equally found that a similar CT-severity score (0%, < 10%, 10–25%, 25–50%, 50–75%, > 75%) upon admission to the hospital was a strong predictor for ICU admittance and in-hospital mortality. They also noted that patient sex, age, and a higher CT-severity score were significantly associated with a prolonged hospital stay. In addition, Grégory et al. [19] found that the extension of pulmonary parenchyma affected by COVID-19 based on real-life visual assessment (< 25%, 25–50%, 50–75%, > 75%) can independently predict negative patient outcome and is correlated with the requirement of artificial aeration and/or death within 14 days of hospitalization. Similar to our results indicating that the optimal cutoff value to predict ICU admission is “Co.V.A.Sc.” unit 3 (26–50% of overall lung parenchyma affected), Guillo et al. [20] also found that in their study, the great majority of patients (68%) who demonstrated > 25% imaging extent of COVID-19 disease were intubated or dead during the first 3 weeks following CT. Likewise, Colombi et al. [21] found that COVID-19 patients with visually assessed well-aerated lung of less than 73% (pneumonia-occupied parenchyma of > 27%) had a 5.4 times greater likelihood of ICU admission or death.

Previous well-known studies have been performed quantifying COVID-19 pneumonia burden on chest CT, nevertheless using different techniques than in the abovementioned literature. Li et al. [22] evaluated the disease extent by assessing the 5 pulmonary lobes separately, while Yang et al. [23] divided the lungs in a total of 20 bronchopulmonary segments and then separately assessed each one of them for the presence of COVID-19 opacities. The authors suggested that visual quantification can reflect the clinical classification of COVID-19 disease [22] and that the CT score is higher in severe than it is in mild cases [23]. Since high CT scores in these studies correspond to worse clinical classifications/ severe cases, we assume that they would most likely correlate to worst patient outcome as well, hence paralleling our results. Assessing and scoring each lobe and each bronchopulmonary segment separately may provide a more accurate and detailed estimate of the disease burden in comparison to assessing both lungs at their entirety; however, it can prove time-consuming and impractical in everyday working conditions where the quantity of CTs being performed can be overwhelming. Nonetheless, employing CT percentage scaling systems such as the “Co.V.A.Sc.” seems to be easier and faster as the observer evaluates both lungs as a whole, without having to individually and separately consider each lobe and/or each bronchopulmonary segment. On this note, in the present study, we found that the mean time necessitated for the evaluation of a chest CT according to the “Co.V.A.Sc.” was 20.98 ± 10.07 s (range: 7–51 s) for observer 1 and 22.02 ± 8.62 s (range: 9–46 s) for observer 2. Additionally, the interclass correlation coefficient (ICC) in our study was 0.944 (95%CI 0.929–0.956) for interobserver agreement, which is comparable to that described in the aforementioned studies of Li et al. (ICC value of 0.976 among the observers) [22] and Yang et al. (ICCmedian = 0.925, ICCmean = 0.936) [23]. On the contrary, these studies did not assess the intraobserver agreement which in our study was also excellent (0.979 (95%CI 0.963–0.988)). Overall, a comparative study assessing the speed and accuracy of the various available CT scoring systems as well as their role in patient management and outcome would be of interest in order to alleviate any discrepancies between centers and observers in the evaluations of chest CTs performed in COVID-19-affected patients.

On a secondary note, we documented an overall in-hospital mortality of 10.98% (30/273 patients) and a total of 9.15% (25/273 patients) requiring ICU admission. These findings are in keeping with available literature showing that COVID-19 pneumonia has an in-hospital mortality ranging from 11 to 15% and that mechanical aeration may be required in up to 17% of hospitalized patients [24, 25]. Our data did not demonstrate a statistically significant correlation between “Co.V.A.Sc.” and patient comorbidities; however, existing literature has already suggested that the presence of risk factors (i.e., arterial hypertension, diabetes, coronary artery disease) carries a poor prognosis and that the more the comorbidities, the worst the clinical outcome [26]. Additionally, no significant correlation was demonstrated in our analysis between the “Co.V.A.Sc.,” the number of days with clinical symptoms prior to CT, PO2/FiO2 ratio upon admission, and the laboratory blood tests.

Limitations of our study include its single-center nature and the fact that it is retrospective; thus, some data might have been missed. Specifically, although all patients included received treatment according to the same international protocols, detailed data regarding different treatment options was not available. As a result, we cannot reliably exclude minor deviations from these protocols in a small fraction of patients. Considering the relatively large patient pool included in our study, we would like to believe that these possible minor deviations would not have affected the results in a significant manner. Moreover, reliable data regarding the vaccination status of all the patients was not available. Nevertheless, roughly 3–4% of the total population of the country had been vaccinated by 17/03/2021 (end of study’s period). As a result, the authors speculate that vaccination would not have affected the results in a significant way. Another limitation is that all patients included in our study were RT-PCR confirmed upon imaging, while in everyday clinical practice, radiologists may often have to provide diagnosis for chest CTs obtained in patients with suspected COVID-19 disease prior to laboratory confirmation. Additionally, imaging was performed within 48 h of hospital admission regardless of the onset of symptoms, therefore creating inconsistency in the patient sample. Furthermore, as is the case with similar research, disease extent in this study is objectively quantified by visual assessment performed by an observer instead of being evaluated by a subjective method (i.e., automated quantification which has also been proven to foresee severe clinical outcome as previously mentioned [21]). Still, high consistency was noted between the evaluations of the observers in this and similar studies mentioned above.

Conclusion

In conclusion, the “Co.V.A.Sc.” is a rudimentary quantitative tool, rapidly stratifying RT-PCR-confirmed COVID-19 patients according to the overall amount of COVID-19-associated pulmonary opacities detected on admission chest CT (0%, 1–10%, 11–25%, 26–50%, 51–75%, > 75%). Evaluating admission chest CTs of RT-PCR-confirmed COVID-19 patients based on the “Co.V.A.Sc.” and comparable quantification tools seems to predict ICU admission and in-hospital death, therefore optimizing risk-stratification and patient management.