A digital twins enabled underwater intelligent internet vehicle path planning system via reinforcement learning and edge computing

Abstract

The Autonomous Underwater Glider (AUG) is a kind of prevailing underwater intelligent internet vehicle and occupies a dominant position in industrial applications, in which path planning is an essential problem. Due to the complexity and variability of the ocean, accurate environment modeling and flexible path planning algorithms are pivotal challenges. The traditional models mainly utilize mathematical functions, which are not complete and reliable. Most existing path planning algorithms depend on the environment and lack flexibility. To overcome these challenges, we propose a path planning system for underwater intelligent internet vehicles. It applies digital twins and sensor data to map the real ocean environment to a virtual digital space, which provides a comprehensive and reliable environment for path simulation. We design a value-based reinforcement learning path planning algorithm and explore the optimal network structure parameters. The path simulation is controlled by a closed-loop model integrated into the terminal vehicle through edge computing. The integration of state input enriches the learning of neural networks and helps to improve generalization and flexibility. The task-related reward function promotes the rapid convergence of the training. The experimental results prove that our reinforcement learning based path planning algorithm has great flexibility and can effectively adapt to a variety of different ocean conditions.

Keywords

• Digital twins
• Reinforcement learning
• Edge computing
• Underwater intelligent internet vehicle
• Path planning