Personalized rehabilitation approach for reaching movement using reinforcement learning

Abstract

Abstract Musculoskeletal disorders challenge significantly the performance of many daily life activities, thus impacting the quality of life. The efficiency of the traditional physical therapy programs is limited by ecological parameters such as intervention duration and frequency, number of caregivers, geographic accessibility, as well as by subjective factors such as patient’s motivation and perseverance in training. The implementation of VR rehabilitation systems may address these limitations, but the technology still needs to be improved and clinically validated. Furthermore, current applications generally lack flexibility and personalization. A VR rehabilitation game simulation is developed, which focuses on the upper-limb movement of reaching, an essential movement involved in numerous daily life activities. Its novelty consists in the integration of a machine learning algorithm, enabling highly adaptive and patient-customized therapeutic intervention. An immersive VR system for the rehabilitation of reaching movement using a bubble popping game is proposed. In the virtual space, the patient is presented with bubbles appearing at different locations and is asked to reach the bubble with the injured limb and pop it. The implementation of a Q-learning algorithm enables the game to adjust the location of the next bubble according to the performance of the patient, represented by his kinematic characteristics. Two test cases simulate the performance of the patient during a training program of 10 days/sessions, in order to validate the effectiveness of the algorithm, demonstrated by the spatial and temporal distribution of the bubbles in each evolving scenario. The results show that the algorithm learns the patient’s capabilities and successfully adapts to them, following the reward policy dictated by the therapist; moreover, the algorithm is highly responsive to kinematic features’ variation, while demanding a reasonable number of iterations. A novel approach for upper limb rehabilitation is presented, making use of immersive VR and reinforcement learning. The simulation suggests that the algorithm offers adaptive capabilities and high flexibility, needed in the comprehensive personalization of a rehabilitation process. Future work will demonstrate the concept in clinical trials.

Keywords

• Adaptive rehabilitation
• Virtual reality
• Reinforcement learning
• Reaching movement
• Serious games