Unmanned surface vehicle escape strategy based on hybrid sampling deep Q-network

Abstract

ObjectiveAiming at the encirclement tactics adopted by enemy ships, this study focuses on the problem of planning an escape strategy when an unmanned surface vehicle (USV) is surrounded by enemy ships. MethodsA hybrid sampling deep Q-network (HS-DQN) reinforcement learning algorithm is proposed which gradually increases the playback frequency of important samples and retains a certain level of exploration to prevent it from falling into local optimization. The state space, action space and reward function are designed to obtain the USV's optimal escape strategy, and its performance is compared with that of the deep Q-network (DQN) algorithm in terms of reward and escape success rate. ResultsThe simulation results show that using the HS-DQN algorithm for training increases the escape success rate by 2% and the convergence speed by 20%. ConclusionsThe HS-DQN algorithm can reduce the number of useless explorations and speed up the convergence of the algorithm. The simulation results verify the effectiveness of the USV escape strategy.

Keywords

• usv
• apollonius circle
• pursuit−evasion
• deep reinforcement learning
• hybrid sampling