Reinforcement Learning for Multiple Daily Injection (MDI) Therapy in Type 1 Diabetes (T1D)

Abstract

In this study, we propose a closed-loop insulin administration framework for multiple daily injection (MDI) treatment using a reinforcement learning (RL) agent for insulin bolus therapy. The RL agent, based on the soft actor–critic (SAC) algorithm, dynamically adjusts insulin dosages based on real-time glucose readings, meal intakes, and previous actions. We evaluated the proposed strategy on ten in silico patients with type 1 diabetes undergoing MDI therapy, considering three meal scenarios. The results show that, compared to an open-loop conventional therapy, our proposed closed-loop control strategy significantly reduces glucose variability and increases the percentage of time the glucose levels remained within the target range. In particular, the weekly mean glucose level reduced from 145.34 ± 57.26 mg/dL to 115.18 ± 7.93 mg/dL, 143.62 ± 55.72 mg/dL to 115.28 ± 8.11 mg/dL, and 171.63 ± 49.30 mg/dL to 143.94 ± 23.81 mg/dL for Scenarios A, B and C, respectively. Furthermore, the percent time in range (70–180 mg/dL) significantly improved from 63.77 ± 27.90% to 91.72 ± 9.27% (p = 0.01) in Scenario A, 64.82 ± 28.06% to 92.29 ± 9.15% (p = 0.01) in Scenario B, and 58.45 ± 27.53% to 81.45 ± 26.40% (p = 0.05) in Scenario C. The model also demonstrated robustness against meal disturbances and insulin sensitivity disturbances, achieving mean glucose levels within the target range and maintaining a low risk of hypoglycemia, which were statistically significant for Scenarios B and C. The proposed model outperformed open-loop conventional therapy in all scenarios, highlighting the potential of RL-based closed-loop insulin administration models in improving diabetes management.

Keywords

• blood glucose management
• reinforcement learning
• multiple daily injection therapy
• simulation for clinical application
• in silico validation