Introduction

Shock is common in critically ill and injured patients.1,2 Patient survival in shock states is dependent on rapid restoration of tissue oxygenation.3,4 Therapeutic restoration of tissue oxygenation involves complex hemodynamic strategies in which cardiac output (CO) optimization plays an important role.5,6,7 Low CO is associated with significant adverse events including death, and therapeutic intervention can improve these outcomes.8,9 Traditionally, clinical CO measurements used pulmonary artery catheter (PAC)-based thermodilution,10 but PAC use has decreased because of risk of complications.10

The development of minimally invasive monitors of CO arose from beliefs in the benefits of CO monitoring and manipulation while reducing the harms of PAC.11 Numerous minimally invasive monitors of CO are commercially available, and uncertainty exists as to which is “best.”12 Three systems with the greatest market share are FloTrac™ (Edwards Lifesciences Corp, Irvine, CA, USA), PiCCOplus™ (Pulsion Medical Systems, Munich, Germany), and CardioQ-ODM™ (Deltex Medical Limited, Chichester, UK), each employing a different technology.13 These devices have been validated against the PAC under limited hemodynamic conditions.14,15 The PAC is a poor reference standard, making it a suboptimal comparator.16,17 Further, most studies do not report results at various flow states (i.e., all, high, or low). This is potentially important as CO monitoring is likely most important at low-flow states, and an aggregate all flow analysis, even if reassuring, may fail to accurately inform clinicians of device performance at critical CO states. Moreover, few studies have investigated CO trending using a gold standard reference device.

Despite the enthusiasm for CO-guided hemodynamic management, the literature does not robustly support its clinical benefit over standard invasive arterial blood pressure-guided strategies.18,19,20 Many practitioners do not routinely monitor CO and continue to use arterial blood pressure guided hemodynamic optimization in high-risk patients.11 This later strategy may also be due to the optimistic assumption that arterial blood pressure and CO are closely related.21,22,23 Despite the limited correlation of blood pressure and CO, guiding therapy by invasive mean arterial pressure (iMAP) may be as efficacious as minimally invasive monitors of CO.

Given the interest in CO-guided hemodynamic management, we sought to evaluate 1) the reliability, bias, precision, interchangeability, and trending (correlation) of the three mentioned minimally invasive monitors, and 2) the reliability and trending of mean iMAP. Both objectives used a gold standard comparator (aortic flow probe), were measured over a wide range of hemodynamic conditions in a pig model, and were analyzed at all flow, low flow ( > 1 standard deviation [SD] below the mean), and high flow ( > 1 SD above the mean) states.

Methods

This prospective interventional study was approved by the University of Saskatchewan Animal Research Ethics Board (AUP 20130131 in February 2014 and 20170023 in May 2017) and the study protocol adhered to the Canadian Council on Animal Care guidelines for humane animal use. This report adheres to the ARRIVE guidelines for animal research.24

Bias considerations of the study design

To minimize bias, all measurements were objective and concurrent. A convenience sample of 12 pigs was chosen as this was affordable and feasible within our institution. The data analysis plan was modified post hoc. We planned to use polar plots, but published recommendations suggest simple regression is satisfactory for comparison with a gold standard.14

Animals

Healthy male pigs (Camborough/PIC Boar 327) median (interquartile range) weight, 30(26–32) kg were used. We used a porcine model because of its similarities with human cardiovascular anatomy and physiology.25 All animals were group housed in a climate-controlled room with 12-hr light-dark cycles with free access to standard food and water. They were fasted eight hours prior to the experimentation but were not denied water.

Anesthesia

The pigs were initially sedated with intramuscular ketamine (5 mg·kg−1) and midazolam (0.5 mg·kg−1). Intravenous access was established, and alfaxalone (mean [SD], 2.5 [0.8] mg·kg−1) was administered intravenously to effect and the pigs’ tracheas were intubated with appropriately sized cuffed endotracheal tubes. All animals were positive pressure ventilated by a Merlin Small Animal Ventilator in 100% oxygen (Vetronic Services Ltd, Torquay, England). Anesthesia was maintained with intravenous remifentanil (20 µg·kg−1·hr−1) in combination with either isoflurane (end tidal concentration, 0.8–1.2%) in oxygen or propofol (12–15 mg·kg−1·hr−1). Initial ventilation settings included a tidal volume of 15 mL·kg−1 and a rate of 12 breaths·min−1 then titrated to achieve an arterial oxygen saturation > 94% and a partial pressure of carbon dioxide of 40–50 mm Hg. A multichannel physiologic monitor (Datex-Ohmeda CardiocapTM/5, GE Healthcare, Helsinki, Finland) was used to monitor electrocardiography, arterial oxygenation, heart rate, blood pressure (systolic, mean, and diastolic), respiratory rate, tidal volume, minute volume, end-tidal carbon dioxide level, and expired isoflurane concentration. A femoral artery was percutaneously cannulated with a 5F 20-cm thermistor-tipped aortic catheter, connected to the PiCCOplus monitoring system for measurement of invasive arterial pressure, and slave-connected to the physiologic monitor. An internal jugular vein was surgically exposed and anterograde cannulated with a 7F 16-cm three-lumen catheter (Arrow; Teleflex, Wayne, PA, USA), to allow for continuous recording of central venous pressure and central venous blood gas sampling. Temperature was continuously monitored via the PiCCOplus monitoring system. Hypothermia was avoided by covering exposed skin and by active forced air warming.

Cardiac output monitoring

Aortic root flow (gold standard)

After appropriate positioning, a left-sided thoracotomy was performed. The aortic root was dissected free and separated from the pulmonary artery. A pre-calibrated appropriately sized ultrasonic transit-time flow probe (PAU-SERIES COnfidence Flowprobes® with Ultrafit Liners 20 or 24 mm, Transonic Systems Inc., Ithaca, NY, USA; accuracy, ± 2%) was positioned without constriction on the aortic root for recording of instantaneous flow and connected to its flow meter (TS420 Perivascular Flow Module). The flow meter’s analogue display and signal quality indicator were used to assess and ensure appropriate CO measurement.

Less invasive CO monitors

All the minimally invasive CO monitors require input of demographic data. The minimal age, weight, and height varies according to monitor type and associated disposables. The DP12 probe with a CardioQ-ODM monitor had the most restrictive low input settings: age, 16 yr; height, 149 cm; and weight, 30 kg. For consistency, we inputted these settings for all animals in all devices.

PiCCOplus (CO measured by arterial pulse contour analysis calibrated to transpulmonary thermodilution)

The previously cannulated femoral artery catheter was connected to the PiCCOplus monitoring system calibrated by the average of three consecutive measurements of thermodilution by injecting 10 mL of iced saline randomly throughout the respiratory cycle into the central venous catheter. This calibration was done immediately prior to experimentation, and six hours later if required.

FloTrac (CO measured by uncalibrated arterial pulse contour analysis)

A brachial artery was cannulated with a 20-G catheter. This catheter was connected to the FloTrac sensor kit and coupled to a Vigileo™ monitor (Edwards Lifesciences, Irvine, CA, USA) to evaluate CO.

CardioQ-ODM (CO measured by esophageal doppler ultrasound)

The probe connected to the monitoring system was advanced into the esophagus until blood flow signals were detected. Before and after each experiment, the probe was repositioned until an ideal sharp velocity outline with narrow spectral dispersion was obtained. Device output was visually monitored throughout experimentation.

Invasive mean arterial blood pressure

Invasive mean arterial blood pressure was measured and recorded via the PiCCOplus monitoring system.

Measurements

Continuous measurements of invasive blood pressure (systolic, mean, and diastolic) as well as CO from the aortic flow probe, FloTrac, PiCCOplus, and CardioQ-ODM were recorded throughout the experiment. After baseline measurements were made and between each of the following experiments, the animal was allowed to recover for 30 min.

Experiments

Effect of increasing CO and afterload

A continuous infusion of epinephrine was administered. The initial dose was 0.1 µg·kg−1·min−1, increased to 0.2 µg·kg−1·min−1, increased again to 0.3 µg·kg−1·min−1, then stopped. Each infusion dose was continued until stability in CO and blood pressure was achieved and continued for an additional five minutes.

Fluid loading

A rapid bolus of 20 mL·kg−1 Ringer’s Lactate solution was administered intravenously. Cardiac output data were continuously collected until stability in CO and blood pressure was achieved.

Effect of increasing CO

A continuous infusion of sodium isoproterenol was administered. The initial dose was 0.01 µg·kg−1·min−1, increased to 0.03 µg·kg−1·min−1, increased again to 0.1 µg·kg−1·min−1, and then stopped.

Effect of decreasing cardiac afterload

A continuous infusion of sodium nitroprusside was administered. The initial dose was 0.3 µg·kg−1·min−1, increased to 0.6 µg·kg−1·min−1, increased again to 0.9 µg·kg−1·min−1, and then stopped.

Effect of decreasing CO

Intravenous boluses of esmolol (2 and then 4 mg·kg−1) were administered to decrease CO. The 2 mg·kg−1 dose was administered first, followed 15 min later with the 4 mg·kg−1 dose.

Hypovolemia

Each animal was phlebotomized with two 20 mL·kg−1 of blood draws. Data were continuously collected until stability in CO and blood pressure were achieved and continued for an additional five minutes; then the second blood draw commenced. At the conclusion of this experiment, the animal was euthanized with a lethal dose of pentobarbital and the experiment terminated.

Data acquisition

Analogue flow signals from the aortic flow meter were digitized at 100 Hz with an analog-to-digital converter (National Instruments USB-6259 multifunction DAQ, National Instruments, Austin, TX, USA), and sampled by customized data logging software (Biobench, National Instruments) operating on a personal computer for online display and storage of data. The CO values from the FloTrac, PiCCOplus, and CardioQ-ODM were copied to computer files by proprietary programs. The iMAP was measured with the PiCCOplus monitor, recording the average mean blood pressure for sequential five second intervals, and copied to computer files by its proprietary program.

Data analysis

The commercial CO monitors produced printouts of their measurements from proprietary programs at different rates, averaging the CO from epochs of different durations: PiCCOplus (1-sec epochs); Flotrac (20-sec epochs); and CardioQ-ODM (30-sec epochs). These data were used at the original output rate for regression against the flow probe. Flow probe data were averaged for epoch lengths corresponding to each device.

Statistical analysis

To simplify results for the reader, data from all pigs was analyzed together as if it were a single end-to-end experiment. We justified this on the basis of the pigs being of the same age, sex, and strain, and having similar weights. To adjust for the differences in CO attributable to different pig sizes or physiology, we adjusted the data by normalizing the mean flow for each pig to the overall mean flow and used the normalization factor for each pig to adjust the point-by-point CO output of each device for each pig before aligning the device outputs end to end. Statistical calculations were performed with SIGMAPLOT version 13.0 (Jandel Scientific, Div. of Jandel Corp., San Rafeal, CA, USA). Because of the large number of comparisons, the statistical significance level was set at P < 0.01. Primary data were tested for normality by the Kolmogorov–Smirnov test and compared with the use of non-parametric Kruskal–Wallis analysis of variance on ranks. We conducted the analyses at all flows > 1 L·min−1, low-flow ( > 1 SD below the mean) and high-flow ( > 1 SD above the mean) states. Categorical variables were analyzed with Chi square, except when comparing data containing a category with zero incidence (iMAP failure), in which case McNemar's test was used.

Function: failure analysis (reliability of data output)

We compared the frequency of failure of the devices (epochs when no CO data were produced) using Chi square and McNemar’s test. Analysis was done for all outputs of each device that occurred during flow outputs > 0.1 L·min−1 as well as at low, medium, and high CO states as measured by the flow probe. Ninety-nine percent confidence intervals (CIs) are reported.

Bias (accuracy) and limits of agreement (precision)

Because the CO devices all incorporate demographics (height, weight, and age) into the CO measurement using confidential proprietary algorithms, and because porcine anatomic proportions are different from those in humans, we considered that we could not assume comparable bias among devices. Thus, we calculated overall limits of agreement from the normalized data. Cardiac output data were not normally distributed, so non-parametric analysis was done to estimate bias and precision with Olofsen’s method and calculator.26,27 Ninety-nine percent CIs are reported.

Percent error (device interchangeability with flow probe)

Percent error was calculated from normalized absolute differences of each CO datum from the corresponding flow reading as follows:

$$100\left(\sum_{0}^{n}\left(\frac{\left|{diff}_{n}\right|}{{Flow}_{n)}}\right)\,\div\,n\right)$$

Ninety-nine percent CIs are reported.

Regression (trending)

Correlation coefficients were calculated with linear regression of each device upon flow probe CO using data output when the device was functioning and flow was > 0.1 L·min−1. Correlation coefficients were compared by the method of Steiger.28 Ninety-nine percent CIs are reported.

Results

Cardiac output from all experimental conditions was collected for ten of 12 animals. Data for animal 1 were not analyzed because of an unrecognized flow probe data acquisition storage failure. FloTrac data for animal 7 were not collected because of an unplanned data storage failure. Table 1 shows the number of paired measurements for the various monitoring modalities for the data in aggregate and at high and low flow states. A representative plot from animal 5 shows the sequence of experimental interventions and measures from all three studied minimally invasive CO monitoring devices (Fig. 1). Time plots for each animal are presented in the Electronic Supplementary Material (ESM) (eFig 1a–i). The results of all animals are summarized in ESM eFig. 1k.

Table 1 Number of paired measurements for the various monitoring modalities and failure rates
Full size table
Fig. 1
figure 1

Representative time plot from animal 5. Flow probe measured cardiac output under labelled experimental conditions. X-axis: time in minutes; Y-axis: normalized cardiac output in L·min−1.

Full size image

None of the CO measures were normally distributed. The flow meter measured flows ranged from 0 to 9.1 L·min−1; the median [1st to 99th percentile] CO over all measurements was 3.2 [1.0 to 5.9] L·min−1. Cardiac outputs less than 2.2 L·min−1 were considered low flow while those greater than 4.4 L·min−1 were considered high flow, representing flows > 1 SD below and above the mean, respectively. None of the minimally invasive monitors of CO consistently provided output at all times (percent failed measurements), although the aortic root flow probe did (Table 1). The PiCCOplus had the fewest failures at all flows and low flows (4.7%; 99% CI, 4.5 to 4.9) and (6.9%: 99% CI, 6.5 to 7.3) (all comparisons P < 0.001). At all flow and low-flow states, iMAP always (with zero failures) provided data (P < 0.001) (Table 1).

The measures of accuracy and precision are presented in Table 2 and Fig 2a–c (Bland–Altman plots). Interchangeability (percent error), and trending ability (correlation coefficients) for each monitoring device at all flow, low flow, and high flow are presented in Tables 3 and 4. The PiCCOplus was most interchangeable with the flow probe at all flow states (20%; 99% CI, 19 to 22) (P < 0.001); at low flows, the CardioQ-ODM (43%; 99% CI, 32 to 63) and Flotrac (45%; 99% CI, 33 to 70) interchangeability were similar (P = 0.07) and both superior to PiCCOplus (48%; 99% CI, 42 to 60) (all comparisons, P < 0.001). Regarding CO trending, the CardioQ-ODM (0.82; 99% CI, 0.81 to 0.83) was statistically superior to the other devices at all flows, but at low flows, iMAP (0.58; 99% CI, 0.58 to 0.60) was best (all comparisons, P < 0.001). Scatterplots for low-flow states for each device and iMAP are shown in Fig 3 a–d while plots for all flow states can be found in the supplement in ESM eFig 2a–d. Four quadrant plots of all devices at all- and low-flow states can be found in ESM eFig 3a–h.

Table 2 Bias and precision of non-invasive cardiac output devices
Full size table
Fig 2
figure 2 figure 2

a–c Bland–Altman plots for each studied monitor of cardiac output: PiCCOplus (2a), FloTrac (2b), CardioQ-ODM (2c).

Full size image
Table 3 Percent error for non-invasive cardiac output devices
Full size table
Table 4 Tracking: a comparison of coefficients of correlation among non-invasive cardiac output monitoring devices and invasive mean arterial pressure
Full size table
Fig 3
figure 3 figure 3

a–d Scatter plots for each studied monitor in low-flow states: PiCCOplus (3a), Flotrac (3b), CardioQ-ODM (3c), and invasive mean arterial pressure (3d).

Full size image

Discussion

This study compared the most commonly used minimally invasive CO monitors to a gold standard ultrasonic aortic root flow probe over a large range of COs (0 to 9.1 L·min−1). None of the studied devices nor iMAP gave useful information under all study conditions. At all flows and when functioning, percent error was lowest with PiCCOplus, then CardioQ-ODM, and greatest with FloTrac. PiCCOplus also failed least often, while FloTrac failed less frequently than CardioQ-ODM. Crucially, at low flow, none of the CO measurement devices performed well, and iMAP trended CO change better than the assessed devices did.

The CardioQ-ODM appeared to provide good graphical signals at low CO when judging the output by the device’s manual, yet the monitor reported failure. This suggests the CardioQ-ODM was measuring CO signal, but the computer algorithm should be updated to provide reliable data at low CO.

The primary objective of this study was to assess the accuracy, precision, interchangeability, and trending ability of the most commonly used minimally invasive monitors of CO over a wide range of hemodynamic conditions; the strengths of our study reflect this goal. The aortic flow probe is described as the gold standard comparator as it provides direct continuous flow measurement, independent of rotation, hemoglobin concentration, or electrically charged molecules.29 We followed the recommendations of a widely quoted paper on methodology for assessing trending ability.14 In pigs, the anatomical and physiologic characteristics of the cardiovascular system are similar to those in humans.25Furthermore, the experimental conditions created grossly reflect the hemodynamics in a wide range of clinically encountered disease states. The extremely invasive nature of the study precludes similar study in humans.

The level of performance needed from a cardiac monitor to improve clinical outcomes is unknown. Despite this, ± 30% percent error (PE) is a widely quoted threshold for acceptable agreement to consider CO devices interchangeable.30 At first consideration, both the CardioQ-ODM and PiCCOplus devices would seem to meet this threshold; however, this initial analysis is flawed for two major reasons. First, the 30% threshold is based on PAC-based CO measurements as the comparator device and incorporates the PAC error into overall error. The flow probe error is negligible; therefore, errors can be attributed to the devices studied alone.29,31 In this situation, the magnitude of acceptable error is less clear, but some authors have suggested that an accuracy of 10% is desirable as this is the error of non-invasive arterial blood pressure measurements;32 others have suggested 15% as this is similar to successive PAC CO measurements;33,34 and still others arbitrarily suggested 20%.35 Using even the most liberal of these thresholds, all the devices fall short. The second problem of accepting 30% PE is the PiCCOplus and CardioQ-ODM met this only at all flow states, whereas clinicians are more likely concerned about CO assessment at low-flow states. Although our definition of low flow is arbitrary (CO < 1 SD below the mean), the best performing device at low flow (CardioQ-ODM) had a PE of 43%, not meeting any described definition of acceptable. Two of the monitoring systems tested rely solely on examination of peripheral pulse data (FloTrac and PiCCOplus) while one (CardioQ-ODM) relies solely on central flow as measured indirectly with ultrasound. Early work found that the accuracy of minimally invasive CO measurement is greatly enhanced by combining central and peripheral information.36

Further, all the devices frequently failed to provide any data output, especially at low-flow states. Guidance on how to interpret these failure rates is not available, but the range of 20–60% failure seems excessive. The issue of monitor output failure is likely more problematic than simply a lack of data output. These devices fail more at low-flow states, but also fail at medium and high flows. Thus, failure may or may not indicate a low-flow state. Hence, device failure increases cognitive load for the care provider, uncertainty of appropriate response, and distraction from other tasks.

Trending ability is another important measurement for CO assessment.14,37 The ability to accurately detect change in CO as an indication of deterioration or response to intervention may be more important than absolute accuracy. It is unclear how to clinically interpret regression analysis of CO monitoring. Although both the CardioQ-ODM and PiCCOplus are highly correlated at all flow states, none of the monitors are well correlated at low-flow states.38

Although the study of the relationship between blood pressure and CO is not new, it remains interesting.22,39 Our rigorous results (116,957 paired measurements) confirm previous data that iMAP has a low positive correlation with CO at all flow states.22Our results show iMAP to be more correlated to CO than any of the studied devices at low-flow states.

A limitation of the study is that young animals were used (~ ten weeks old). These animals presumably did not suffer from adult human diseases including cardiovascular disease, potentially limiting the generalization of our results. Additionally, the animals weighed much less than an average adult. We chose a young porcine model for several reasons: 1) the known similarities of human and porcine cardiovascular anatomy and physiology,25 2) the expense of using a primate animal model, and 3) the danger to investigators of using adult pigs, which can weight more than 300 kg.

Another limitation of our study relates to the various experimental hemodynamic conditions created to mimic physiologic stress and disease states. It is likely the hemodynamic effects of most disease states, especially septic shock, are far more complex than those we created, limiting the generalizability of our results to sick humans. Additionally, the relevance of our analysis at various flow states could be questioned as the clinician would not know the true flow state and therefore could not apply our flow state findings to observed device measurements. This criticism is valid if these devices were used in isolation. We contend that in clinical practice, CO monitoring would rarely be used as a sole monitor, rather would be used in conjunction with numerous other monitors in addition to the context of the clinical scenario. The clinicians’ integration of the totality of the available data would likely influence the assessment of patient flow state and therefore application of our results. Further, should a device not perform well at critical low-flow states, we contend that its utility is limited regardless of its performance at other or aggregate flow states. Finally, we did not evaluate all commercially available minimally invasive monitors of CO.

Numerous studies have assessed the performance of minimally invasive CO devices using the PAC as a comparator. For studies investigating CO compared with a gold standard under a wide range of hemodynamic conditions, the body of literature is much smaller.35,40,41,42,43,44

Acute care physicians often work on the premise that achieving adequate CO is an essential management component for care of critically ill patients especially considering that signals of instability, need for intervention, and resuscitation end points are often clinically challenging. The logical extension of this management framework is that the use of CO monitors should improve patient outcomes. A systematic review suggests that CO-guided hemodynamic management does not improve mortality, but may reduce important morbidities and hospital length of stay (the underlying data are of limited quality).20 The knowledge that survivors of major surgical procedures have higher CO45 and, likely more importantly, the belief that monitoring and manipulating CO should improve patient outcomes, continues to drive intensive research and commercial interest. The reasons for weak improvements in clinical outcomes are probably numerous, but poor monitor performance may be an important factor; our results suggest that there is significant room for improvement in this domain. Further, the studies investigating CO-guided hemodynamic management compared it with either goal directed or usual care, both primarily based on arterial blood pressure measurement. This general lack of outcome improvement could be explained knowing that the minimally invasive CO devices perform similar to iMAP monitoring under low-flow conditions.

Summary

The PiCCOplus performed better than other devices when considering all CO states, but none of the studied minimally invasive monitors of CO performed well, especially at critical low-flow states in a young pig model. This evidence points to possible poor performance in humans. None of the studied monitors tracked change in CO better than mean arterial blood pressure at low flow states.