Neural radiance fields assisted by image features for UAV scene reconstruction

Abstract

Abstract With the rapid advancement of Unmanned Aerial Vehicle applications, vision-based 3D scene reconstruction has demonstrated significant value in fields such as remote sensing and target detection. However, scenes captured by UAVs are often large-scale, sparsely viewed, and complex. These characteristics pose significant challenges for neural radiance field (NeRF)-based reconstruction. Specifically, the reconstructed images may suffer from blurred edges and unclear textures. This is primarily due to the lack of edge information and the fact that certain objects appear in only a few images, leading to incomplete reconstructions. To address these issues, this paper proposes a hybrid image encoder that combines convolutional neural networks and Transformer to extract image features to assist NeRF in scene reconstruction and generate new perspective images. Furthermore, we extend the NeRF architecture by introducing an additional branch that estimates uncertainty values associated with transient regions in the scene, enabling the model to suppress dynamic content and focus on static structure reconstruction. To further improve synthesis quality, we also refine the loss function used during training to better guide network optimization. Experimental results on a custom UAV aerial imagery dataset demonstrate the effectiveness of our method in accurately reconstructing and rendering UAV-captured scenes.