Height estimation from single aerial imagery using contrastive learning based multi-scale refinement network

Abstract

Height map estimation from a single aerial image plays a crucial role in localization, mapping, and 3D object detection. Deep convolutional neural networks have been used to predict height information from single-view remote sensing images, but these methods rely on large volumes of training data and often overlook geometric features present in orthographic images. To address these issues, this study proposes a gradient-based self-supervised learning network with momentum contrastive loss to extract geometric information from non-labeled images in the pretraining stage. Additionally, novel local implicit constraint layers are used at multiple decoding stages in the proposed supervised network to refine high-resolution features in height estimation. The structural-aware loss is also applied to improve the robustness of the network to positional shift and minor structural changes along the boundary area. Experimental evaluation on the ISPRS benchmark datasets shows that the proposed method outperforms other baseline networks, with minimum MAE and RMSE of 0.116 and 0.289 for the Vaihingen dataset and 0.077 and 0.481 for the Potsdam dataset, respectively. The proposed method also shows around threefold data efficiency improvements on the Potsdam dataset and domain generalization on the Enschede datasets. These results demonstrate the effectiveness of the proposed method in height map estimation from single-view remote sensing images.

Keywords

• height estimation
• aerial imagery
• digital surface models
• contrastive learning
• local implicit constrain