Incremental Dual Heuristic Dynamic Programming Based Hybrid Approach for Multi-Channel Control of Unstable Tailless Aircraft

Abstract

Actor-critic based online reinforcement learning control has been proved to be promising method for control of aerial vehicles. However, it is difficult to guarantee high-level success rate of initial training and to tune the large amount of parameters for actors and critics considering unstable multi-input and multi-output (MIMO) aircraft. In order to facilitate and simplify the training of the actor and critic for unstable aircraft, classic stability augmentation system (SAS) is designed for the open-loop aircraft first. Then the online incremental model based dual heuristic dynamic programming (IDHP) method, which has been proposed recently, is extended in application to design a multi-channel robust adaptive controller, and MIMO form network structures are designed and determined for the actors and critics considering the three-channel coupling issues. Consequently, the classic SAS and the IDHP controller make up a novel hybrid control framework. In this control framework, the SAS takes charge of counteracting the unstable eigenvalues of the open-loop aircraft system, and the IDHP takes charge on guaranteeing robust and adaptive performance for high-performance tailless aircraft equipped with the SAS. Specifically, the introduction of the classic control method decreases the difficulty of the initial training for multi-channel IDHP controller. The tuning process for initial parameters of actor and critic neural networks in multiple channels is greatly facilitated. Without the help of SAS, the initial training for multi-channel IDHP controllers of unstable plants is almost impossible to succeed. Finally, the novel hybrid control architecture and method are validated using the Innovative Control Effectors (ICE) model, which has unstable modes in the longitudinal dynamics. Typical aerodynamic model uncertainties are numerically simulated to demonstrate the effectiveness of the proposed control method.

Keywords

• Incremental DHP
• actor and critic
• unstable aircraft
• reinforcement learning
• MIMO control