Feedback-Based Curriculum Learning for Collision Avoidance

Abstract

This paper proposes a novel curriculum learning approach for collision avoidance using feedback from the deep reinforcement learning (DRL) training process. Previous research on DRL-based collision avoidance algorithms has encountered challenges such as long training times and difficulty in convergence due to sparse rewards. To address these issues, curriculum learning has been used to divide the target task into multiple subtasks for training. However, manual or random curriculum design often generates unnecessary subtasks that do not improve performance. Furthermore, a standardized curriculum design method for collision avoidance has not yet been presented. Therefore, this paper introduces a curriculum-based collision avoidance learning method that utilizes feedback during the training phase. The proposed method differs from traditional curriculum learning in that the subtask is not predetermined before training. Instead, the curriculum is modified during training based on feedback obtained from validation environments. If a robot demonstrates high collision avoidance performance in a validation environment, it is then validated in more challenging environments for rigorous evaluation. Conversely, if collision avoidance performance is low in the validation environment, the robot is trained in a new environment to overcome frequent collision situations. Simulations and practical experiments were conducted for the proposed method, which showed better performance compared to the non-curriculum method.

Keywords

• Collision avoidance
• curriculum learning
• deep reinforcement learning
• subtask generation