DQN-Based Shaped Reward Function Mold for UAV Emergency Communication

Abstract

Unmanned aerial vehicles (UAVs) have emerged as pivotal tools in emergency communication scenarios. In the aftermath of disasters, UAVs can be communication nodes to provide communication services for users in the area. In this paper, we establish a meticulously crafted virtual simulation environment and leverage advanced deep reinforcement learning algorithms to train UAVs agents. Notwithstanding, the development of reinforcement learning algorithms is beset with challenges such as sparse rewards and protracted training durations. To mitigate these issues, we devise an enhanced reward function aimed at bolstering training efficiency. Initially, we delineate a specific mountainous emergency communication scenario and integrate it with the particularized application of UAVs to undertake virtual simulations, constructing a realistic virtual environment. Furthermore, we introduce a supplementary shaped reward function tailored to alleviate the problem of sparse rewards. By refining the DQN algorithm and devising a reward structure grounded on potential functions, we observe marked improvements in the final evaluation metrics, substantiating the efficacy of our approach. The experimental outcomes underscore the prowess of our methodology in effectively curtailing training time while augmenting convergence rates. In summary, our work underscores the potential of leveraging sophisticated virtual environments and refined reinforcement learning techniques to optimize UAVs deployment in emergency communication contexts.

Keywords

• unmanned aerial vehicle UAV
• deep Q-learning network DQN
• reward shaping