Optimization Control of Attitude Stability for Hexapod Robots Based on Reinforcement Learning

Abstract

To enhance the attitude stability of hexapod robots on rough terrain, this paper introduces a layered modular motion control method, termed reinforcement learning integrated with foot impedance control and an optimized reward mechanism. It integrates gait and foot impedance coordinated planning with reinforcement learning. By constructing a discrete gait model and a state space, the gait planning problem is transformed into an optimal sequence decision-making problem based on stability margin, which can be solved using the Markov decision process. Within the reinforcement learning framework, the robot autonomously optimizes gaits and foot trajectories using feedback from foot-ground contact states and base inclination. Our designed reward mechanism enhances gait adaptation, significantly augmenting stability and traversal efficiency. The algorithm accelerates convergence and ensures reliable decision-making by compressing the action space to four dimensions, reducing the need for internal sensor information, thus enhancing practicality. Comprehensive simulation experiments demonstrate that this method significantly outperforms central pattern generator methods in terms of distribution of the roll and pitch angles, distance traveled, number of successes, and mean reward.

Keywords

• Hexapod robot
• hierarchical motion
• reinforcement learning
• stability optimization
• terrain adaptation