Advancing Active Suspension Control With TD3-PSC: Integrating Physical Safety Constraints Into Deep Reinforcement Learning

Abstract

This study addresses the limitations of traditional active and semi-active suspension control systems in terms of adaptability and nonlinear handling, by exploring the potential of Deep Reinforcement Learning (DRL) techniques. Initially, a framework based on the Twin Delayed Deep Deterministic policy gradient (TD3) specific to active suspension systems was developed. Building on this, an enhanced TD3 algorithm, TD3-PSC (Physically Safe Constraint TD3), incorporating physical safety constraints was proposed. The TD3-PSC algorithm extends the state space to enhance understanding of suspension dynamics and improve adaptability. To accommodate the physical constraints and actuator characteristics inherent in suspension systems, TD3-PSC introduces guided training with real physical constraints and employs immediate termination and high penalty mechanisms to ensure safety and practicality of the algorithm. The simulation results demonstrate that TD3-PSC significantly outperforms the linear quadratic regulator (LQR), deep deterministic policy gradient (DDPG), and standard TD3 baseline, achieving improvements in control performance of 73.81%, 43.72%, and 32.14% under standard Class C road conditions, respectively. Additionally, it exhibits excellent generalization capabilities.

Keywords

• Deep reinforcement learning
• TD3
• active suspension system
• physical constraint