Closed-Loop Control of Fluid Resuscitation Using Reinforcement Learning

Abstract

Fluid resuscitation (therapy) is used to maintain tissue perfusion and restore cardiac functions in critical care. Automated fluid therapy can result in faster care, fewer dosing errors, and less cognitive burden on healthcare providers, ultimately improving patient outcomes. Despite a few attempts at automating this process, fluid management is an open research area for which optimal, personalized strategies are yet to be developed. This work presents a novel, model-free, subject-specific dose adjustment tool for fluid resuscitation. The proposed approach is based on reinforcement learning (RL) where a Q-learning algorithm automatically recommends subject-specific fluid infusion dosages in different hemorrhaging scenarios without having the knowledge of dose-response models. Comparison studies against two model-free fluid resuscitation controllers, i.e., fuzzy and proportional-integral-derivative (PID), within a verified simulated environment demonstrated the superior performance of the proposed approach in the closed-loop control of fluid resuscitation. Statistical analyses of performance measures indicated that the RL approach, with lower average resuscitation rates, can achieve more desired mean arterial pressure (MAP) responses than the fuzzy and PID controller for all virtual subjects. Additionally, simulation results demonstrated the higher robustness of our approach than the other two methods against external disturbances in resuscitation scenarios. These results confirm the potential of RL in the closed-loop control of hemodynamic responses in fluid therapy.

Keywords

• Automated fluid resuscitation
• fluid management
• mean arterial pressure (MAP)
• model-free reinforcement learning
• Q-learning