Autonomous decision-making of UAV cluster with communication constraints based on reinforcement learning

Abstract

Abstract Artificial intelligence techniques are increasingly applied in the study of autonomous decision-making in unmanned clustered distributed systems. However, communication constraints has become a big bottleneck that restricts its performance. To address the need for unmanned aerial vehicles(UAVs) to execute collaborative attack missions in complex communication-constrained environments, this paper propose an autonomous decision-making method for UAVs based on Multi-Agent Reinforcement Learning (MARL). Firstly, the autonomous decision-making processes of UAV clusters are modeled as Decentralized Partially Observable Markov Decision Processes(Dec-POMDPs). Next, the algorithm is enhanced within the framework of Multi-Agent Deep Deterministic Policy Gradient(MADDPG) by designing an explicit inter-intelligent communication mechanism to achieve information exchange among UAVs. Subsequently, the algorithm utilizes Long Short-Term Memory(LSTM) networks to process the local observations of the UAVs, enhancing the effectiveness of the information sent by combining historical data with current observations. Finally, multiple rounds of experiments are conducted across various communication-constrained scenarios. Simulation results indicate that the proposed method improves the task completion capability by 46.0% and enhances stability by 24.9% compared to baseline algorithm MADDPG. Additionally, the algorithm demonstrates better generalization and exhibits good scalability, effectively adapting to varying numbers of UAVs. This research provides new theoretical insights and a technical framework for the collaboration of UAVs in environments with communication constraints, which holds great practical importance in improving the ability and application scope of UAV systems.

Keywords

• UAV cluster
• Autonomous decision-making
• Communication constraint
• Reinforcement learning