Adaptive Deep Reinforcement Learning for Efficient 3D Navigation of Autonomous Underwater Vehicles

Abstract

The exploration of the underwater environments has recently accelerated with the development of the Autonomous Underwater Vehicle (AUV). One of the key elements for enhancing the autonomy of AUVs navigation across various applications is efficient path planning. Reinforcement Learning (RL) methods have been successfully introduced for path planning of AUVs, particularly in high-dimensional state spaces, where prior knowledge of the environment is unfeasible. In this work, we propose a Deep Reinforcement Learning (DRL) method for efficient AUV navigation in 3 Dimension (3D) environments, utilizing input from vision sensors to obtain information about the motion of the AUV and the surrounding space. We adopt a multi-tier approach in order to validate the performance of the proposed DRL approach in three different neural network architectures leveraging on adaptation and accuracy, with path length, execution time and success of operation being considered as the optimization objectives. Finally, a simulation platform is built to evaluate the performance of the proposed method, with experimental results showcasing enhanced decision-making capability for the AUV navigation, which translates to a higher level of autonomy for the vehicle in unknown environments.

Keywords

• AUV
• underwater
• actor-critic algorithm
• AUV navigation
• dynamic path planning
• proximal policy optimization