Data analyzed in this study were part of the Medicare National Pneumonia Project, a component of Centers for Medicare and Medicaid Services' Quality Improvement Program. Eligible patients were fee-for-service Medicare beneficiaries who had been discharged from the hospital with a principal diagnosis of pneumonia (International Classification of Disease, Ninth Edition, Clinical Modification [ICD-9-CM] codes of 480.0 through 483.99, 485 through 486.99, or 487) (20) or a principal diagnosis of septicemia or respiratory failure (ICD-9-CM codes 038.XX or 518.81) and a secondary diagnosis of pneumonia, as defined previously. Patients were included regardless of whether they were eligible for Medicare by virtue of age, disability, or end-stage renal disease. A derivation cohort and a validation cohort were used. For the derivation cohort, discharges within each state were sampled for a 6-month period between April 1998 and March 1999. From 346,105 pneumonia discharges during these sampling periods, we randomly selected up to 850 cases per state, for a total sample of 39,242 cases. The validation cohort was derived in a similar fashion from hospital discharges between July 1, 2000 and March 31, 2001.

Each medical record was subjected to explicit review using an electronic data collection instrument with predefined instructions. Detailed information including patient demographics, comorbidities, physical and laboratory findings, type and timing of antibiotic treatment, and selected outcomes were collected by experienced abstractors in the Medicare Clinical Data Abstraction Center. Most of the clinical variables were derived from the PSI of Fine and coworkers (14). Patients were considered as having received antibiotics before blood cultures if they had received systemic antibiotics any time during the week before presentation or at the hospital before the drawing of blood cultures. Comorbid illnesses were defined on the basis of a documented history as opposed to exam or laboratory findings. For example, the finding of liver disease was based on the documentation of any of 14 descriptors of chronic liver disease. To test data reliability, 957 charts were subjected to duplicate abstraction. The κ statistic was calculated for pneumonia confirmation and exclusion, clinical characteristics, and blood culture results. The κ statistic ranged from 0.58 (chest radiograph showed pneumonia) to 1.00 (transfer from another acute care hospital) for exclusion criteria, from 0.45 (liver disease) to 0.96 (systolic blood pressure) for clinical characteristics, and 0.84 for blood culture results.

Cases were included if the admission note demonstrated that the physician's working diagnosis was pneumonia at the time of admission and if there was a confirmatory chest radiograph indicating a new infiltrate. Cases were excluded if missing data elements prevented analysis, if the patient was transferred from another acute care facility, was discharged on the day of admission, or was discharged from an acute care facility during the previous 14 days. If there had been more than one admission for pneumonia during the sampling period, only the first was included. Patients were defined as bacteremic if a blood culture drawn within 36 hours of presentation to the hospital grew an organism not defined as a contaminant. The 36-hour cutoff was chosen to maximize the number of cases and avoid including patients with nosocomial bacteremia. Although Enterococcus sp. rarely, if ever, cause pneumonia, and many cases of bacteremia due to nonpneumococcal Streptococci are not due to pneumonia, we opted not to exclude patients with bacteremia due to these organisms, as the aim of this study was to identify what factors predict bacteremia in patients with a clinical syndrome diagnosed as community-acquired pneumonia. In such patients, the finding of bacteremia is likely important even if the diagnosis of pneumonia is not always accurate.

Because standard practice is to draw two sets of blood cultures for patients with pneumonia, only the results of the first two blood cultures during this time period were considered. Blood cultures that grew coagulase-negative staphylococci, Corynebacterium sp. (other than jekieum), Clostridium sp., Micrococcus sp., Propionibacterium sp., and Bacillus sp. were defined as contaminated.

Summary statistics were calculated for the abstracted data. Measures of association including univariate odds ratios and χ² tests were performed. The relationship between bacteremia and demographics, medical comorbidity, physical examination, laboratory and radiographic variables, as well as the PSI (14) were determined by univariate analysis. Variables associated with bacteremia with p values less than 0.15 were used in a multivariate logistic regression analysis to determine which factors were independently associated with bacteremia (excluding contaminants). To identify the problem of collinearity, we calculated the φ coefficient of two dichotomous variables (21). If two independent variables were highly correlated, the variable with the largest variance was excluded from the multivariate analysis. Logistic regression analysis was performed using the backward elimination procedure. The goodness of fit of the model was tested with the Hosmer–Lemeshow test (22), which revealed adequate model fit (p = 0.39).

Once we developed a prognostic model based on findings in the derivation cohort, we applied the model to the validation cohort to evaluate the predictive power of the model. Receiver operating characteristic (ROC) curves were constructed by a series of cut points from both the derivation and validation cohorts (23). To test if the model prediction is better than chance prediction, we calculated the area under ROC curves and their 95% confidence intervals (95% CI) for derivation cohort (area under ROC curve = 0.68; 95% CI, 0.66–0.70) and validation cohort (area under ROC curve = 0.68; 95% CI, 0.66–0.70) (24). To test the reliability of the model, we compared the area under ROC curves between the two cohorts (p = 0.94).

All reported p values are based on two-tailed tests. Statistical significance was accepted at p values less than 0.05. Analyses were conducted using the software packages Statistical Analysis System (SAS version 8.0; SAS Institute, Cary, NC). ROC curve analysis was performed using AccuROC software (AccuROC version 2.5; Montreal, PQ, Canada).

After validating the model, we created a prediction tool to define the risk of bacteremia, using clinical criteria that would be available to clinicians when the decision regarding blood cultures was being made. All variables that were independently associated with the presence of bacteremia with an odds ratio of greater than 1.3 were considered for inclusion in the tool. Patients were stratified according to the number of predictive factors present and divided into three risk categories (low, moderate, and high).

We then developed a decision support tool to select patients for blood culture performance on the basis of the estimated risk of bacteremia. An a priori assumption was that any predictive model would need to identify at least 85% of bacteremias to be clinically useful. Because performing one blood culture resulted in a sensitivity for bacteremia of approximately 0.8 when compared with two blood cultures, we considered tools wherein we recommended only one blood culture for patients without a high risk of bacteremia. In these cases, the result of the first blood culture drawn was used to determine if the tool would have detected this patient's bacteremia. Thus, we designed a tool whereby the number of blood cultures collected was proportional to the risk of bacteremia.

Of the 39,242 pneumonia cases in the 1998 to 1999 sampling period, 16,327 (41.6%) were excluded from the primary analyses because no blood cultures were drawn during the 36 hours after presentation to the hospital. Of the remaining 22,915 cases, 5,180 were excluded because of one of the general exclusion criteria, whereas another 4,692 were excluded because of missing data elements (most commonly, the timing of blood cultures relative to initial antibiotics). Thus, 13,043 cases were included in the derivation cohort. The initial sample size and frequency of exclusions were similar for the validation cohort. Demographic and clinical characteristics as well as outcomes were similar for each cohort (Table 1)

.

Bacteremia was detected in 7% of the derivation cohort and 7% of the validation cohort, whereas 5% of all patients had at least one contaminated blood culture (Table 2)

. Patients who had a contaminated blood culture had a longer length of hospital stay compared with patients with negative blood cultures, 7.7 ± 6.1 days versus 6.6 ± 6.6 days, respectively, for the derivation cohort and 7.8 ± 10.7 days versus 6.5 ± 5.7 days, respectively, for the validation cohort (p < 0.01 for both cohorts). Multivariate analysis revealed that much of this excess length of stay was attributable to the false-positive blood culture (0.8 ± 0.3 days for the derivation cohort and 1.0 ± 0.3 days for the validation cohort). Patients with contaminated blood cultures were more frequently treated with vancomycin at some point during their hospital stay in both the derivation cohort (17 vs. 6%) and the validation cohort (19 vs. 6%) (p < 0.0001 for both cohorts).

Table 3

shows the characteristics associated with bacteremia that were used in the prediction rule. The use of antibiotics before blood cultures, either before presentation or in the healthcare facility was negatively associated with the detection of bacteremia (odds ratio, 0.5; 95% CI, 0.5–0.6). Liver disease, three vital sign abnormalities, and three laboratory abnormalities were found to be highly associated with bacteremia (Table 3). Several variables not independently associated with bacteremia were noteworthy. These included age and several variables relating to the degree of respiratory compromise, such as the presence of hypoxemia, pleural effusion, or the number of lobes involved radiographically. A respiratory rate of 30 or above was associated with bacteremia (odds ratio, 1.3; 95% CI, 1.1–1.5; p = 0.002) but not highly enough to be useful in the predictive model. The PSI was not significantly associated with bacteremia when Groups I, II, and III were compared with Groups IV and V (odds ratio, 0.9; 95% CI, 0.7–1.1; p = 0.2).

Table 4

demonstrates that the risk of bacteremia could be predicted by assessing the prior use of antibiotics and the number of clinical predictors present. In the derivation cohort, 2% of the low-risk patients were bacteremic, 5% of the moderate-risk patients were bacteremic, and 11% of the high-risk patients were bacteremic. Results were similar in the validation cohort. Table 5 demonstrates the number of patients with bacteremia who would be detected by the prediction tool and number of blood cultures that would be avoided by its use. Use of the tool would have resulted in the detection of 88 to 89% of patients with bacteremia but would have required the performance of 37 to 39% fewer blood cultures. Among patients with the highest risk of mortality, those in PSI Class V, use of the tool would have allowed detection of 94% of patients with bacteremia in each cohort. We calculated the performance of our prediction tool for patients younger than 65 years of age and found its sensitivity among both cohorts for these younger patients was 89%.

We explored the performance of a simplified model that excluded the three laboratory predictors of bacteremia, as such a model might be easier to use in clinical practice. The decision support tool based on this model was optimized by the performance of one blood culture on all but the highest-risk patients, for whom two blood cultures are recommended. This model identified 86 to 88% of patients with bacteremia and decreased the number of blood cultures by 44% (Table 6)

. However, it only identified 89% of PSI Class V patients with bacteremia.

The pathogen-specific performance of the tool was also explored, with two noteworthy findings. The performance of the tool in detecting pneumococcal bacteremia was higher than its overall performance, as it detected 93% of pneumococcal bacteremias, but the tool was less sensitive for all other Streptococcus sp., detecting only 65% of bacteremias caused by these organisms.

Among the PSI Class IV and V patients in both cohorts whose bacteremia was not detected by the use of this prediction tool, 30-day mortality (20%) was significantly lower than among all patients with bacteremia in these risk classes (29%) (p < 0.05). This suggests that those patients whose bacteremia was not detected by the prediction tool may have been “missed” because they were less seriously ill, despite being bacteremic.

We also examined the group of patients who were excluded from the primary analyses because they had no blood cultures. Sixty-one percent of patients without blood cultures were in PSI risk Class IV or V, whereas 77% of patients who were cultured were in these risk classes. Our model predicted that 6% of those excluded from the derivation cohort were bacteremic, with similar results in the validation cohort. Because 42% of all patients with pneumonia were not cultured, it appears that with current clinical practice, about 38% of patients with bacteremia having pneumonia do not have blood cultures performed.

We have shown that several clinical variables were independently associated with bacteremia in a large cohort of patients admitted to the hospital with pneumonia. Using these predictors, we created a decision support tool that recommends not obtaining blood cultures from patients at low risk of bacteremia, obtaining one blood culture from moderate-risk patients, and obtaining two blood cultures from high-risk patients. When applied to a second large cohort of patients with pneumonia, this tool identified nearly 90% of all patients with bacteremia and 94% of PSI Class V patients, while allowing 38% fewer blood cultures to be drawn. Despite our finding of several clinical variables that were highly associated with bacteremia, many patients without bacteremia also had vital sign and laboratory abnormalities, and this factor limited our ability to further restrict the recommended number of blood cultures.

Prior investigators have recommended that blood cultures not be performed on low-risk patients with community-acquired pneumonia (8–13). Waterer and Wunderink found that by restricting blood cultures to PSI Class IV and V patients, 56% of blood cultures could be avoided, but they would have failed to detect 29% of patients with bacteremia (13). In a cohort of 531 patients with community-acquired pneumonia, Roson and coworkers found that only 67% of the patients with bacteremia were in PSI Class IV and V (25). Campbell and coworkers found little correlation between bacteremia and the PSI (8). We found that the PSI was not an independent predictor of bacteremia. The superior performance of our model in predicting bacteremia is not surprising, as patient age is a major determinant of the PSI (14) and was not an independent risk factor for bacteremia.

What are the implications of failing to diagnose bacteremia in some patients with community-acquired pneumonia? Meehan and coworkers found that the performance of blood cultures was associated with improved 30-day mortality among Medicare patients with pneumonia (6). Our results also demonstrate that organisms other than Strepococcus pneumoniae caused over 60% of bacteremic community-acquired pneumonia, potentially increasing the risks of empiric therapy when the infecting pathogen is not identified. However, several studies have shown that even positive blood culture results usually do not result in adjustments of antibiotic therapy in patients with pneumonia (8, 9, 11). It has also been reported that patients with pneumococcal bacteremia do not require a longer course of antibiotics than patients without bacteremia so that the identification of bacteremia may not be important for determining the length of therapy (26).

The potential risks incurred by the 11% false-negative rate associated with the use of our tool must be compared with the limitations of current practice. Forty-two percent of patients in the Medicare database were not cultured. Consequently, about 38% of patients with bacteremia having pneumonia were not identified. Because this group of patients was less severely ill than the overall population with pneumonia, it seems that some clinicians decide to perform blood cultures based on clinical judgment regarding the severity of illness and/or the likelihood of bacteremia. The use of our tools could allow more accurate targeting of patients likely to be bacteremic.

Any potential risks associated with a decreased use of blood cultures must be balanced against the risks and costs of obtaining the approximately 10,000 cultures required to identify the approximately 100 bacteremias that were missed in each cohort. Given the 3.1% rate of contaminated blood cultures, many more false-positive blood cultures than true positives would occur. Because blood cultures are initially reported as positive before the final identification of the organism occurs, a false-positive blood culture often results in increased pharmacy costs, further testing, and a prolongation of the hospital stay. In our two cohorts, a false-positive culture was independently associated with approximately a 1-day excess length of hospital stay. Prior investigators have estimated that a false-positive blood culture adds approximately $6,000 to the cost of hospitalization, mostly due to a prolongation of the hospital length of stay by as much as 4 to 8 days (17, 18). By targeting only higher risk patients for blood cultures, much of this cost might be averted. We and others have noted a higher frequency of vancomycin use as a result of false-positive blood cultures, and this phenomenon may contribute to the continuing increase in resistance to this antibiotic (27).

We recommend the use of one blood culture for patients with a moderate risk of bacteremia. Although others have recommended the use of only one blood culture in certain populations (28), many experts question the usefulness of one blood culture in any setting (29). However, in a patient with a clinical presentation suggesting community-acquired pneumonia who has no indwelling intravascular devices, even a single blood culture growing an organism that commonly contaminates blood cultures should be easily identifiable as a contaminant because none of these organisms cause pneumonia.

Our study has some limitations. There are clinical factors associated with bacteremia that were not measured. Examples might be injection drug use or physical exam evidence suggesting disseminated infection. In patients defined as low risk by our rule with clinical signs suggesting bacteremia, blood cultures should still be performed. Another potential limitation is that our database was derived from Medicare patients and may not be generalizable to younger patients. However, patients below 65 years of age made up approximately 10% of each cohort, and the prediction tool performed equally well for these patients. Also, the variables we used were dichotomous variables, and it is possible that a model using continuous variables may have yielded different results. Finally, although there would be indisputable benefits derived from reducing the total number of blood cultures drawn from patients with community-acquired pneumonia, there was a 20% mortality rate among patients with bacteremia who would have been missed by this decision tool. We cannot dismiss the possibility that these severely ill patients could be at increased risk if their bacteremia was not identified. Recent reports suggest that the magnitude of this risk may be small, given the lack of evidence of an affect on patient outcomes (8, 10, 11). However, we must await further studies to determine whether outcomes would be optimized by striving to identify 100% of bacteremias or whether the appropriate balance of risk and benefit would be achieved by obtaining even fewer blood cultures and only identifying, for example, 75% of bacteremias.

There may be barriers to the adoption of these decision support tools in the clinical arena. The use of laboratory values is a potential limitation to the use of the original tool, as this could delay the performance of blood cultures and hence the delivery of antibiotics. However, a common practice in Emergency Departments is to place an intravenous catheter and simultaneously draw routine laboratories and, if appropriate, at least one set of blood cultures for febrile patients who appear as if they may require hospitalization. Furthermore, if laboratory results are not available, blood cultures can be obtained and the decision as to whether to send them for processing can be made once all of the necessary data are available. Of course, this concern would not be a factor if the simplified tool were used, as it did not rely on laboratory data. Another practical matter that could affect the use of these tools is that many physicians may not make the effort to commit either of them to memory. Although they are much simpler than the PSI, it is possible that they would not be extensively used unless incorporated into pneumonia management protocols with preprinted orders.

In summary, we found that several easily determinable clinical characteristics are independently associated with bacteremia in patients admitted to the hospital with pneumonia. Using these characteristics, we developed a model that accurately predicted the likelihood of bacteremia at the time of presentation to the hospital. Two decision support tools based on this model were more sensitive than the PSI in predicting bacteremia. We suggest that the use of one of these tools would reduce the number of blood cultures performed on patients with community-acquired pneumonia, yet allow the identification of nearly 90% of patients with bacteremia. Avoiding blood cultures in low-risk patients would minimize the excess cost and prolonged length of hospital stay associated with false-positive blood cultures.