A Study on Autonomous Control of Underwater Manipulator Autonomous Operation Based on Deep Reinforcement Learning

Abstract

Owing to the inherent complexities of underwater environments, coupled with restrictive observational angles, the precise operation of an underwater manipulator during autonomous tasks is a sizable undertaking. To tackle this issue, this paper proposes a method for autonomous control of an underwater manipulator, leveraging the robust adaptive capacity of reinforcement learning algorithms. Initially, a reinforced learning approach is developed using proximal policy optimization (PPO) intertwined with an actor-critic (AC) algorithm to formulate the autonomous control strategy. Subsequently, an artificial potential field-based reward shaping method is introduced to address the sparse reward predicament apparent throughout the training process. Lastly, a simulation experiment validates the devised control strategy trained using the aforementioned procedures. The verification results illustrate that the strategy competently converges and commands the autonomous movement of the underwater manipulator towards intended targets. It allows for quick, fluid transitions and provides a smooth, stable track for the end effector.

Keywords

• underwater manipulator
• reinforcement learning
• reward shaping
• autonomous operation