Large Language Model Guided Reinforcement Learning Based Six-Degree-of-Freedom Flight Control

Abstract

As artificial intelligence (AI) technology advances rapidly, its increasing involvement in military defense fosters intelligent air combat domain development. The Intelligent Flight Controller (IFC) is a crucial technology and foundation for intelligent air combat decision-making systems. Controlling 6 Degree-of-freedom (DOF) aircraft in close-to-real-world environments requires an adaptable and dynamic decision-making controller. Most IFC researches focus on simplistic flight trajectory design and validation, while air combat requires aircraft that can perform complex tactical maneuvers. Deep reinforcement learning (DRL) provides a suitable technical paradigm. However, DRL suffers from sparse rewards, insufficient supervisory signals, low sampling efficiency, and slow convergence. In contrast, Large Language Model (LLM) possesses abundant knowledge about the real world, contextual understanding, and reasoning capabilities. By leveraging this, LLM can serve as prior knowledge for DRL, thereby reducing DRL training time. This paper proposes an LLM-guided deep reinforcement learning framework for IFC, which utilizes LLM-guided deep reinforcement learning to achieve intelligent flight control under limited computational resources. LLM provides direct guidance during training based on local knowledge, which improves the quality of data generated in agent-environment interaction within DRL, expedites training, and offers timely feedback to agents, thereby partially mitigating sparse reward issues. Additionally, we present an effective reward function to comprehensively balance the aircraft coupling control to ensure stable, flexible control. Finally, simulations and experiments show that the proposed techniques have good performance, robustness, and adaptability across various flight tasks, laying a foundation for future research in the intelligent air combat decision-making domain.

Keywords

• Intelligent flight control
• large language model
• deep reinforcement learning
• 6 DOF aircraft