AoI-Aware Resource Management for Smart Health Via Deep Reinforcement Learning

Abstract

The freshness of information is critical for patient vital signs and physiological parameters in the healthcare system because changes in these parameters can indicate a patient’s overall health status and guide treatment decisions. In this paper, we consider an edge device-aided smart healthcare system that relies on a resource management scheme. The medical center requires patient information, and edge nodes process the latest measurements received by each wearable device. Our goal is to find the optimal strategy to minimize the worst case of information freshness, i.e., the peak AoI age of information (PAoI). Firstly, we model the problem as a Markov Decision Process (MDP). Then, we design two separate Reinforcement Learning (RL)-based algorithms to find the optimal strategy that minimizes energy consumption and the average PAoI. To minimize energy consumption, we propose a pair of sleep mechanisms, including the $N$ policy and $p$ wake-up policy, to improve the energy efficiency of each wearable device. Simulation results show that the proposed wake-up strategy and the proposed RL algorithm make a better trade-off between the average PAoI and power dissipation compared to the baseline schemes.

Keywords

• Smart health
• age of information (AoI)
• sleep-scheduling
• deep reinforcement learning (DRL)
• deep deterministic policy gradient (DDPG)