Optimization of swimming mode for elongated undulating fin using multi-agent deep deterministic policy gradient

Abstract

Optimizing speed and propulsive efficiency are the most crucial survival skills for biomimetic robots. This paper investigates a swimming mode controller inspired by the black Knifefish to govern the fast-swimming gait with high propulsive efficiency for an elongated undulating fin robot. The proposed swimming mode controller is composed of a couple of Hopf oscillator-based central pattern generators (CPG) to generate the moving gait of robotic fish and a novel variant of Reinforcement Learning (RL) known as Multi-Agent Deep Deterministic Policy Gradient (MA-DDPG) for optimizing the propulsive efficiency. The proposed swimming controller facilitates the autonomous optimization of the oscillatory amplitude of the robotic fish to improve its propulsive efficiency. The proposed MA-DDPG demonstrates an aptitude for functioning within mixed cooperative-competitive environments. Furthermore, it effectively mitigates the drawback of zero amplitude in the updating process of conventional reinforcement learning (RL) methodologies. These findings highlight the potential utility of the MA-DDPG in optimizing the performance of multi-agent systems in dynamic, real-world scenarios. The simulation results show that the undulating fin robot reaches a maximum thrust of 0.9 N with a propulsive efficiency of 12.48 %, which is higher than that of traditional reinforcement learning methods.

Keywords

• Central pattern generators (CPGs)
• Multi-agent deep deterministic policy gradient
• Hopf oscillator
• Reinforcement learning
• Robotic fish