DRL-Based Improved UAV Swarm Control for Simultaneous Coverage and Tracking with Prior Experience Utilization

Abstract

Area coverage and target tracking are important applications of UAV swarms. However, attempting to perform both tasks simultaneously can be a challenge, particularly under resource constraints. In such scenarios, UAV swarms must collaborate to cover extensive areas while simultaneously tracking multiple targets. This paper proposes a deep reinforcement learning (DRL)-based, scalable UAV swarm control method for a simultaneous coverage and tracking (SCT) task, called the SCT-DRL algorithm. SCT-DRL simplifies the interaction between UAV swarms into a series of pairwise interactions and aggregates the information of perceived targets in advance, based on which forms the control framework with a variable number of neighboring UAVs and targets. Another highlight of SCT-DRL is using the trajectories of the traditional one-step optimization method to initialize the value network, which encourages the UAVs to select the actions leading to the state with less rest time to task completion to avoid extensive random exploration at the beginning of training. SCT-DRL can be seen as a special improvement of the traditional one-step optimization method, shaped by the samples derived from the latter, and gradually overcomes the inherent myopic issue with the far-sighted value estimation through RL training. Finally, the effectiveness of the proposed method is demonstrated through numerical experiments.

Keywords

• deep reinforcement learning
• simultaneous coverage and tracking
• multi-UAV control
• distributed method