Autonomous Quadrotor Path Planning Through Deep Reinforcement Learning With Monocular Depth Estimation

Abstract

Autonomous navigation is a formidable challenge for autonomous aerial vehicles operating in dense or dynamic environments. This paper proposes a path-planning approach based on deep reinforcement learning for a quadrotor equipped with only a monocular camera. The proposed method employs a two-stage structure comprising a depth estimation and a decision-making module. The former module uses a convolutional encoder-decoder network to learn image depth from visual cues self-supervised, with the output serving as input for the latter module. The latter module uses dueling double deep recurrent Q-learning to make decisions in high-dimensional and partially observable state spaces. To reduce meaningless explorations, we introduce the Insight Memory Pool alongside the regular memory pool to provide a rapid boost in learning by emphasizing early sampling from it and relying on the agent's experiences later. Once the agent has gained enough knowledge from the insightful data, we transition to a targeted exploration phase by employing the Boltzmann behavior policy, which relies on the refined Q-value estimates. To validate our approach, we tested the model in three diverse environments simulated with AirSim: a dynamic city street, a downtown, and a pillar world, each with different weather conditions. Experimental results show that our method significantly improves success rates and demonstrates strong generalization across various starting points and environmental transformations.

Keywords

• Deep reinforcement learning
• experience replay
• insight memory pool (IMP)
• path planning
• Q-learning
• autonomous aerial vehicle (AAV)