Deep reinforcement learning-based model-free path planning and collision avoidance for UAVs: A soft actor–critic with hindsight experience replay approach

Abstract

In this paper, we propose a soft actor–critic (SAC) algorithm with hindsight experience replay (HER), called SACHER, which is a class of deep reinforcement learning (DRL) algorithm. SAC is an off-policy model-free DRL algorithm that outperforms earlier DRL algorithms in terms of exploration and robustness. However, in SAC, maximizing the entropy-augmented objective degrades the optimality of learning outcomes. We propose SACHER to improve the learning performance of SAC. We apply SACHER to the path planning and collision avoidance control of unmanned aerial vehicles (UAVs). We demonstrate the effectiveness of SACHER in terms of the success rate, learning speed, and collision avoidance performance of UAV operation.

Keywords

• Deep reinforcement learning
• Soft actor–critic
• Hindsight experience replay
• UAV path planning
• Collision avoidance and control