B-APFDQN: A UAV Path Planning Algorithm Based on Deep Q-Network and Artificial Potential Field

Abstract

Deep Q-network (DQN) is one of the standard methods to solve the Unmanned Aerial Vehicle (UAV) path planning problem. However, the way agent deepens its cognition of the environment through frequent random trial-and-error leads to slow convergence. This paper proposes an optimized DQN with Artificial Potential Field (APF) as prior knowledge called B-APFDQN for path planning. Replacing the traditional neural network which has only one Q-value output with a multi-output neural network to promote the training process in combination with APF. Furthermore, a SA- $\varepsilon $ -greedy algorithm that can automatically adjust the stochastic exploration frequency with steps and successes is proposed in order to prevent the agent from falling into local optimum. We remove the nodes that do not affect the path connectivity and apply the B-spline algorithm to make the path shorter and smoother. Simulation experiments show that the proposed B-APFDQN algorithm performs better than the classical DQN, has a strong ability to avoid falling into local optimum, and the obtained paths are smooth and shorter, which proves the effectiveness of B-APFDQN in the UAV path planning problem.

Keywords

• Reinforcement learning
• deep Q-network
• artificial potential field
• path planning