A deep deterministic policy gradient method for collision avoidance of autonomous ship

Abstract

This research addresses the crucial problem of collision avoidance decision making for autonomous ships under diverse encounter situations. Building upon the Deep Deterministic Policy Gradient (DDPG) algorithm, appropriate reward functions based on the International Regulations for Preventing Collisions at Sea (COLREGS) have been designed to effectively guide intelligent agents in acquiring optimal strategies. By incorporating the concept of potential reward shaping, the proposed approach ensures efficient obstacle avoidance while adhering strictly to the established rules. Moreover, extensive simulations have been conducted to validate the algorithm’s performance in collision avoidance for both dual-ship and multi-ship scenarios under varying encounter situations, and a comparative analysis with the TD3 algorithm has been undertaken. The obtained results demonstrate that the proposed algorithm exhibits rapid convergence and stable training performance. The resulting models successfully achieve collision-free navigation while strictly adhering to the COLREGS. Particularly, in two-ship encounter situations, the proposed algorithm outperforms the trajectory planned by the TD3 algorithm in terms of shorter path length and higher efficiency.

Keywords

• unmanned surface vehicle|autonomous navigation obstacle avoidance|deep reinforcement learning|colregs