Deep reinforcement learning and decoupling proportional-integral-derivative control of a humanoid cable-driven hybrid robot

Abstract

Aiming at the application of robots in service, medical treatment, rehabilitation, and other fields, a humanoid cable-driven hybrid robot by imitating the structure of human arm is designed in this article. The robot is composed of a cable–spool–pulley system, series mechanism, and coaxial spherical parallel mechanism, which can achieve six degrees of freedom movement in space. A challenge with cable driving is that the movement of the rear end joints (such as pitch and roll) can alter the length or tension of the cable driven by the frontend joints, resulting in joint coupling. This interference can lead to a decrease in the motion accuracy of the robotic arm. In addition, it also affects the durability of cables or mechanical components such as bearings and pulleys. Considering the joint motion coupling phenomenon caused by the cable-driven, the decoupling method is proposed, and the kinematic model of the robot is established. To solve the nonlinear coupling characteristics and the uncertainty of the dynamics parameters, a controller is proposed for the humanoid cable-driven hybrid robot, combining proportional-integral-derivative (PID) control based on decoupling method (DC-PID) and the double delay deep deterministic policy gradient (TD3) deep reinforcement learning algorithm. The trajectory tracking of the end-effector position and orientation are controlled by using DC-PID and TD3. And the simulation results show that the proposed control method has good trajectory tracking and convergence performance according to trajectory tracking error and training reward. Finally, the humanoid cable-driven hybrid robot prototype is developed. The experimental results of coupling compensation show that the average maximum error of the joint is reduced by 77.58% after considering the coupling compensation of the joint. The results of the controller validation experiments show that the DC-PID controller reduces the maximum error in each axis by 11.54%, 35.29%, and 40.16%, respectively, compared to the open-loop experiments.