Transient Controller Design Based on Reinforcement Learning for a Turbofan Engine with Actuator Dynamics

Abstract

To solve the problem of transient control design with uncertainties and degradation in the life cycle, a design method for a turbofan engine’s transient controller based on reinforcement learning is proposed. The method adopts an actor–critic framework and deep deterministic policy gradient (DDPG) algorithm with the ability to train an agent with continuous action policy for the continuous and violent turbofan engine state change. Combined with a symmetrical acceleration and deceleration transient control plan, a reward function with the aim of servo tracking is proposed. Simulations under different conditions were carried out with a controller designed via the proposed method. The simulation results show that during the acceleration process of the engine from idle to an intermediate state, the controlled variables have no overshoot, and the settling time does not exceed 3.8 s. During the deceleration process of the engine from an intermediate state to idle, the corrected speed of high-pressure rotor has no overshoot, the corrected-speed overshoot of the low-pressure rotor does not exceed 1.5%, and the settling time does not exceed 3.3 s. A system with the designed transient controller can maintain the performance when uncertainties and degradation are considered.

Keywords

• turbofan engine
• transient control
• reinforcement learning
• deep deterministic policy gradient (DDPG)