Application of Improved Q-Learning Algorithm in Dynamic Path Planning for Aircraft at Airports

Abstract

Guiding and controlling aircraft within an airport is a decision-making process based on safety and efficiency in a highly dynamic and stochastic environment. Currently, many airports rely on manual monitoring and command to provide appropriate taxiing paths for aircraft. With the increasing complexity of airport structures and flight volumes, there is a need for an algorithm that can autonomously search for the shortest taxiing paths while satisfying the specific taxiing regulations and maintaining safe separations between aircraft in a dynamic scenario. We propose an improved approach based on the Q-Learning algorithm, a reinforcement learning method, to provide taxiing path guidance for aircraft. The Q-Learning algorithm exhibits adaptability in dynamic and stochastic environments. However, the traditional Q-Learning algorithm lacks the iteration stability and computational efficiency required in high-dynamic scenarios, and the shortest paths found often fail to meet the requirements due to the specific regulations of airport control. We first make three improvements to the Q-Learning algorithm to address these challenges. These improvements include optimizing Q-table exploration, resetting initial Q-table values, and introducing a dynamic exploration factor to enhance the algorithm’s computational efficiency and accuracy. We also incorporate conflict avoidance strategies related to civil aviation regulations to ensure that the final path adheres to airport control regulations. Finally, we validate the fused and improved algorithm in a gridded airport environment model. Compared to traditional methods, the results demonstrate that the improved algorithm provides more efficient taxiing guidance for aircraft while ensuring operational safety. Furthermore, the algorithm strategically avoids conflicts with other moving aircraft, thereby increasing the utilization of airport taxiing resources.

Keywords

• Aircraft navigation
• artificial intelligence
• reinforcement learning
• dynamic path planning
• processing time and steps
• conflict avoidance