Neural Radiance Fields for High-Resolution Remote Sensing Novel View Synthesis

Abstract

Remote sensing images play a crucial role in remote sensing target detection and 3D remote sensing modeling, and the enhancement of resolution holds significant application implications. The task of remote sensing target detection requires a substantial amount of high-resolution remote sensing images, while 3D reconstruction tasks generate denser models from diverse view perspectives. However, high-resolution remote sensing images are often limited due to their high acquisition costs, a scarcity of acquisition views, and restricted view perspective variations, which pose challenges for remote sensing tasks. In this paper, we propose an advanced method for a high-resolution remote sensing novel view synthesis by integrating attention mechanisms with neural radiance fields to address the scarcity of high-resolution remote sensing images. To enhance the relationships between sampled points and rays and to improve the 3D implicit model representation capability of the network, we introduce a point attention module and batch attention module into the proposed framework. Additionally, a frequency-weighted position encoding strategy is proposed to determine the significance of each frequency for position encoding. The proposed method is evaluated on the LEVIR-NVS dataset and demonstrates superior performance in quality assessment metrics and visual effects compared to baseline NeRF (Neural Radiance Fields) and ImMPI (Implicit Multi-plane Images). Overall, this work presents a promising approach for a remote sensing novel view synthesis by leveraging attention mechanisms and frequency-weighted position encoding.

Keywords

• novel view synthesis
• neural radiance fields
• remote sensing
• attention
• volume rendering