Deep Reinforcement Learning-Based UAV Path Planning Algorithm in Agricultural Time-Constrained Data Collection

Abstract

In the Agricultural Internet of Things (AgIoT), Unmanned Aerial Vehicles (UAVs) can be used to collect sensor data. Thus, UAVs must plan the appropriate data collection paths so that sensors can collect the data under different positions and generate time-constrained data. Therefore, this paper proposes a UAV path planning algorithm based on Deep Reinforcement Learning (DRL), which jointly optimizes location, energy, and time deadline to maximize the data-energy ratio. The path planning process is modeled and decomposed into a Markov Decision Process (MDP), and then a Prioritized Experience Replay Double Deep Q Network (PER-DDQN) model is used to calculate the optimal solution. Furthermore, a time-constrained reward function and an improved adaptive upper confidence bound (UCB) exploration function are proposed to balance exploration and exploitation in the DRL algorithm, affording the developed algorithm to converge quickly and smoothly. The simulations demonstrate that compared with traditional methods, the proposed algorithm presents better path selection during the data collection process, lower execution time, and a higher data-energy ratio. Our algorithm promotes the use of UAV in AgIoT.

Keywords

• adaptive exploration
• deep reinforcement learning
• markov decision process
• path planning
• reward function