RLSF: Multimodal Sleep Improvement Based Reinforcement Learning

Abstract

As informatization 3.0 accelerates the pace of people’s life and work, people’s happiness index and physical and mental health have become the focus of attention and research in sociology, psychology and medicine. Currently, neurological diseases represented by insomnia have become common chronic diseases. However, existing insomnia treatment methods mainly focus on drug therapy and EEG-based expert intervention, ignoring the individual variability of insomnia patients, the high cost of expert intervention, and the privacy of the user’s treatment environment. Therefore, aiming at the effect of white noise lite on sleep quality, this paper proposes a time-frequency domain correlation multimodal sleep enhancement framework based on reinforcement learning (RLSF), which is a closed-loop feedback sleep improvement framework that includes hardware and software. Specifically, the individual sleep state is fed for learning through EEG sensors’ input, and the agent is gradually trained to suit the sleep habit. This paper provides a reinforcement learning environment in which different agents can be deployed easily; then, we propose the Deep Net Sleep Improvement agent (DNSI agent) and the Time and Frequency-based Lightweight Sleep Improvement agent (TFLSI agent) for RLSF. Finally, the substantial experiments compare DNSI and TFLSI agent performance, and the results indicate that these two agents both have decision-making ability, and three volunteers’ Pittsburgh Sleep Quality Index significantly reduces by 3–7 points within two months and the average time to sleep is reduced by 131.4 seconds. Our code is publicly available at https://github.com/TerryZAG/RLSF. Our self-made dataset is publicly available at https://github.com/TerryZAG/TGAM-datasets-for-RLSF.

Keywords

• Sleep improvement
• reinforcement learning
• time-frequency domain correlation