Deep Reinforcement Learning for Model Predictive Controller Based on Disturbed Single Rigid Body Model of Biped Robots

Abstract

This paper modifies the single rigid body (SRB) model, and considers the swinging leg as the disturbances to the centroid acceleration and rotational acceleration of the SRB model. This paper proposes deep reinforcement learning (DRL)-based model predictive control (MPC) to resist the disturbances of the swinging leg. The DRL predicts the swing leg disturbances, and then MPC gives the optimal ground reaction forces according to the predicted disturbances. We use the proximal policy optimization (PPO) algorithm among the DRL methods since it is a very stable and widely applicable algorithm. It is an on-policy algorithm based on the actor–critic framework. The simulation results show that the improved SRB model and the PPO-based MPC method can accurately predict the disturbances of the swinging leg to the SRB model and resist the disturbance, making the locomotion more robust.

Keywords

• biped robots
• single rigid body
• model predictive control
• deep reinforcement learning