Critical Care Explorations (Oct 2024)

Development and Validation of a Machine Learning Model for Early Detection of Untreated Infection

  • Kevin G. Buell, MBBS, MS,
  • Kyle A. Carey, MPH,
  • Nicole Dussault, MD,
  • William F. Parker, MD, PhD,
  • Jay Dumanian, MD,
  • Sivasubramanium V. Bhavani, MD, MS,
  • Emily R. Gilbert, MD,
  • Christopher J. Winslow, MD,
  • Nirav S. Shah, MD, MPH,
  • Majid Afshar, MD, MSCR,
  • Dana P. Edelson, MD, MS,
  • Matthew M. Churpek, MD, MPH, PhD

DOI
https://doi.org/10.1097/CCE.0000000000001165
Journal volume & issue
Vol. 6, no. 10
p. e1165

Abstract

Read online

BACKGROUND:. Early diagnostic uncertainty for infection causes delays in antibiotic administration in infected patients and unnecessary antibiotic administration in noninfected patients. OBJECTIVE:. To develop a machine learning model for the early detection of untreated infection (eDENTIFI), with the presence of infection determined by clinician chart review. DERIVATION COHORT:. Three thousand three hundred fifty-seven adult patients hospitalized between 2006 and 2018 at two health systems in Illinois, United States. VALIDATION COHORT:. We validated in 1632 patients in a third Illinois health system using area under the receiver operating characteristic curve (AUC). PREDICTION MODEL:. Using a longitudinal discrete-time format, we trained a gradient boosted machine model to predict untreated infection in the next 6 hours using routinely available patient demographics, vital signs, and laboratory results. RESULTS:. eDENTIFI had an AUC of 0.80 (95% CI, 0.79–0.81) in the validation cohort and outperformed the systemic inflammatory response syndrome criteria with an AUC of 0.64 (95% CI, 0.64–0.65; p < 0.001). The most important features were body mass index, age, temperature, and heart rate. Using a threshold with a 47.6% sensitivity, eDENTIFI detected infection a median 2.0 hours (interquartile range, 0.9–5.2 hr) before antimicrobial administration, with a negative predictive value of 93.6%. Antibiotic administration guided by eDENTIFI could have decreased unnecessary IV antibiotic administration in noninfected patients by 10.8% absolute or 46.4% relative percentage points compared with clinicians. CONCLUSION:. eDENTIFI could both decrease the time to antimicrobial administration in infected patients and unnecessary antibiotic administration in noninfected patients. Further prospective validation is needed.