Research on Intelligent Maneuvering Decision-Making in Close Air Combat Based on Deep Q Network

Abstract

Aiming at the problem of UCAV maneuvering decision-making in close air combat, the design of reinforcement learning reward function and the selection of hyper-parameters are studied based on the framework of deep Q network algorithm. For the sparse reward problem in reinforcement learning, an auxiliary reward function that considers angle, range, altitude and speed factors is used to describe the air combat mission accurately and guide the learning direction of the agent correctly. Meanwhile, aiming at the problem of applying reinforcement learning hyper-parameter selection, the influence of learning rate, the number of network nodes and network layers on the decision-making system is explored, and a good range of parameter selection is given, which provides a reference for the following research on parameter selection. The simulation results show that the trained agent can learn the optimal maneuver strategy in different air combat situations, but it is sensitive to reinforcement learning hyper-parameters.

Keywords

• |air combat|autonomous maneuvering decision-making|deep reinforcement learning|dqn|reward function|intelligent maneuver