Online Safe Flight Control Method Based on Constraint Reinforcement Learning

Abstract

UAVs are increasingly prominent in the competition for space due to their multiple characteristics, such as strong maneuverability, long flight distance, and high survivability. A new online safe flight control method based on constrained reinforcement learning is proposed for the intelligent safety control of UAVs. This method adopts constrained policy optimization as the main reinforcement learning framework and develops a constrained policy optimization algorithm with extra safety budget, which introduces Lyapunov stability requirements and limits rudder deflection loss to ensure flight safety and improves the robustness of the controller. By efficiently interacting with the constructed simulation environment, a control law model for UAVs is trained. Subsequently, a condition-triggered meta-learning online learning method is used to adjust the control raw online ensuring successful attitude angle tracking. Simulation experimental results show that using online control laws to perform aircraft attitude angle control tasks has an overall score of 100 points. After introducing online learning, the adaptability of attitude control to comprehensive errors such as aerodynamic parameters and wind improved by 21% compared to offline learning. The control law can be learned online to adjust the control policy of UAVs, ensuring their safety and stability during flight.

Keywords

• online safe flight control
• constrained reinforcement learning
• meta-learning