Adaptive Height Optimization for Cellular-Connected UAVs: A Deep Reinforcement Learning Approach

Abstract

Providing reliable connectivity to cellular-connected Unmanned Aerial Vehicles (UAVs) can be very challenging; their performance highly depends on the nature of the surrounding environment, such as density and heights of the ground Base Stations (BSs). On the other hand, tall buildings might block undesired interference signals from ground BSs, thereby improving the connectivity between the UAVs and their serving BSs. To address the connectivity of UAVs in such environments, this paper proposes a Reinforcement Learning (RL) algorithm to dynamically optimise the height of a UAV as it moves through the environment, with the goal of increasing the throughput or spectrum efficiency that it experiences. The proposed solution is evaluated in two settings: using a series of generated environments where we vary the number of BS and building densities, and in a scenario using real-world data obtained from an experiment in Dublin, Ireland. Results show that our proposed RL-based solution improves UAV Quality of Service (QoS) by 6% to 41%, depending on the scenario. We also conclude that, when flying at heights higher than the buildings, building density variation has no impact on UAV QoS. On the other hand, BS density can negatively impact UAV QoS, with higher numbers of BSs generating more interference and deteriorating UAV performance.

Keywords

• Experimental measurements
• massive MIMO
• reinforcement learning
• two-tier networks
• unmanned aerial vehicles (UAVs)