Deep reinforcement learning to control an unmanned swarm system

Abstract

Recently, testing and using micro-unmanned vehicles, such as unmanned aerial vehicles (UAVs), in scenarios such as supply transportation, agricultural management, and military operations have become more common. It is no longer sufficient to control a single UAV to accomplish all missions. With the increasing complexities associated with operating and task requirements, an unmanned swarm requires a series of algorithms with higher efficiency, greater generalization ability, and better adaptability than the earlier algorithms. A combination of unmanned swarms with artificial intelligence is becoming a common solution to manage the above requirements. Deep reinforcement learning (DRL) is a machine learning method that combines deep learning (DL) and reinforcement learning (RL); therefore, this method has the advantages of DL and RL. Using an RL method, an agent can learn from the environment by trial and error and make decisions that autonomously obtain high scores. However, when the given environment is complex, the decision function of the agent may be too difficult to implement and then the agent cannot make the correct decision. The DL method has strong fitting ability. A suitable deep neural network can simulate any linear or nonlinear function. If the DL method is used to simulate the decision function in RL, the hybrid method can solve the problem that an agent cannot solve and make a correct decision in a complex environment. The combination of an unmanned swarm and a DRL method has been widely studied. This paper introduces the concept of DRL from the perspective of principles and characteristics. This paper analyzes several typical DRL algorithms, discusses the various control requirements of a UAV swarm, and then focuses on the achievements of combining DRL and a UAV swarm control. Finally, this paper presents viewpoints on the application prospects and challenges related to landing and transformation in the combination field. The concept of an unmanned swarm originated from the study of the behavior of biological groups. Several species of bees, ants, birds, fish, and other creatures exhibit complex group behaviors. These clusters comprise many independent individuals in accordance with certain aggregation rules to form a coordinated, orderly group movement mechanism. Similar to biological clusters, in the field of robotics or UAVs, unmanned swarm systems are crowded intelligent systems. These systems consist of multiple homogeneous or heterogeneous unmanned equipment to achieve mutual behavior coordination and jointly complete specific tasks through interactive feedback and incentive response of information. In practical applications, an unmanned swarm system needs to meet the requirements of an open environment, a changeable situation, limited resources, and real-time responses. This system needs to have multicore collaborative capabilities such as distributed collaborative perception, intelligent collaborative decision-making, and robust collaborative control. The distributed intelligent collaborative control method based on DRL can fully meet the control requirements of high intelligence and robustness of unmanned cluster systems.

Keywords

• unmanned swarm
• swarm control
• deep reinforcement learning
• multiagent
• artificial intelligence
• swarm intelligence