Maneuver Decision of Cooperative Air Combat within Visual Range Based on MATD3

Abstract

In order to improve the cooperative ability of multiple unmanned combat aircraft vehicle(UCAV) in air combat, a cooperative air combat maneuver decision method based on multi-agent dual delay depth deterministic policy gradient algorithm (MATD3) is proposed. Firstly, the air combat environment is constructed based on the three degree of freedom dynamic model of UCAV, and the action space represented by the change of control quantity is designed based on the pilot’s control mode. Secondly, the design of state space and reward function is optimized to improve the accuracy of maneuvering decision and the safety of maneuvering flight. The relative relationship between friendly aircraft and enemy aircraft is introduced into state space, the continuous reward function is established according to the relative angle, relative distance and other air combat situation factors, and the flight constraints are integrated into the discrete type reward function. Training techniques such as phased training, heuristic guidance, dual exploration mechanism, and alternating freezing game are adopted to improve the convergence speed of the algorithm and the robustness of the maneuvering strategy.Finally, a two-to-one air combat simulation scenario is constructed, and the results show that our two aircraft can show obvious cooperative behavior, which improves the perception and control of air combat situation.

Keywords

• |ucav|cooperativeair combat|maneuver decision|multi-agent|deep reinforcement learning