Path planning method of snake-like robot based on improved DDPG algorithm

Abstract

Aiming at the problems of low training speed and convergence speed caused by falling into a dead zone of traditional reinforcement learning algorithm of the snake-like robot when performing path planning task in multi-obstacle environment, an improved deep deterministic policy gradient(DDPG) algorithm was proposed. Firstly, a multi-layer long short-term memory (LSTM) neural network model was introduced into the actor-critic network to control the memory and forgetting degree of information in the experience pool; secondly, the CPG(central pattern generators) network was integrated into a reinforcement learning model by optimizing feature parameters, designing new network state space and reward function, finally, The improved algorithm and the traditional algorithm were deployed in Webots environment for simulation experiments.The results show that compared with the traditional algorithm, the overall training time of the improved algorithm is reduced by 15% on average, and the number of iterations to reach the target point is reduced by 22% on average, which reduces the times of falling into the dead zone during driving and obviously improves the convergence speed. The algorithm can effectively guide the snake-like robot to avoid obstacles, thus providing a new idea for its performing path planning task in multi-obstacle environment.

Keywords

• robot control; snake-like robot; improved ddpg algorithm; intensive learning; cpg network; webots 3d simulation