Prospective evaluation of a machine learning-based clinical decision support system (ViSIG) in reducing adverse outcomes for adult critically ill patients

Abstract

Introduction: Prospective clinical evaluations of decision support tools for the ICU are almost non-existent. We sought to test whether a novel clinical decision support tool called ViSIG, based on machine learning algorithms, could be delivered via a user interface to clinicians in real-time, and if this led to changes in ICU mortality, length of stay (LOS), readmission, and duration of mechanical ventilation (dMV). Methods: The study took place in six adult ICUs at two hospitals in the U.S. There were two study phases: one where clinicians were blinded to ViSIG scores (ViS), followed by a phase where ViS were visible via a user interface. The patients' ViS were updated every 30 min. ICU mortality, ICULOS, readmission status, and dMV were assessed on each patient in both phases, along with possible severity of illness confounders. Changes in outcomes between phases were evaluated by multivariable models, adjusting for patients’ severity of illness. Results: There were 2256 admissions in the blinded phase, followed by 1890 admissions in the unblinded phase. ICU mortality decreased from the blinded to the unblinded phase, but severity-adjusted mortality was unchanged. Actual and severity-adjusted ICULOS declined significantly from the blinded to unblinded phase (0.38 days). Severity-adjusted readmissions were reduced by 35 %, as was severity-adjusted dMV (0.62 days). Conclusion: Making information from ViSIG visible to clinicians in real-time was statistically associated with severity-adjusted decreases in ICULOS, readmission, and dMV. Future studies are needed to determine whether the decreases in these outcomes are due to changes in treatment delivery resulting from clinician knowledge of patients’ ViSIG scores.

Keywords

• Critical care
• Intensive care units
• Prospective evaluation
• Mortality prediction
• Genetic algorithms
• Optimization