PID Controller Based on Improved DDPG for Trajectory Tracking Control of USV

Abstract

When navigating dynamic ocean environments characterized by significant wave and wind disturbances, USVs encounter time-varying external interferences and underactuated limitations. This results in reduced navigational stability and increased difficulty in trajectory tracking. Controllers based on deterministic models or non-adaptive control parameters often fail to achieve the desired performance. To enhance the adaptability of USV motion controllers, this paper proposes a trajectory tracking control algorithm that calculates PID control parameters using an improved Deep Deterministic Policy Gradient (DDPG) algorithm. Firstly, the maneuvering motion model and parameters for USVs are introduced, along with the guidance law for path tracking and the PID control algorithm. Secondly, a detailed explanation of the proposed method is provided, including the state, action, and reward settings for training the Reinforcement Learning (RL) model. Thirdly, the simulations of various algorithms, including the proposed controller, are presented and analyzed for comparison, demonstrating the superiority of the proposed algorithm. Finally, a maneuvering experiment under wave conditions was conducted in a marine tank using the proposed algorithm, proving its feasibility and effectiveness. This research contributes to the intelligent navigation of USVs in real ocean environments and facilitates the execution of subsequent specific tasks.

Keywords

• unmanned surface vehicles
• PID controller
• reinforcement learning
• DDPG