3-D Model Extraction Network Based on RFM-Constrained Deformation Inference and Self-Similar Convolution for Satellite Stereo Images

Abstract

Traditional three-dimensional (3-D) reconstruction methods for satellite stereo images (SSIs) are limited by observation angles and image resolution, resulting in poor reconstruction results and only a rough 3-D model of the extracted target. Meanwhile, deep-learning methods require a large number of training samples and restoring the complete 3-D structure of the target is challenging when it is quite different from the training sample. To address these problems, we propose a 3-D extraction method for SSIs based on self-similar convolution and a deformation inference network constrained by a rational function model (RFM). Inspired by the implicit relationship between 2-D image features and 3-D shapes, we construct a 2-D–3-D mapping relationship to mine the depth features of remote-sensing images by incorporating the RFM. The deformation result of each point in the point cloud is inferred by a graph convolution network to iteratively optimize the 3-D reconstruction effect of the visible surface. We construct the self-similar convolution module by utilizing the self-similarity characteristics existing in the target itself. The reconstruction results of the invisible surface are optimized while establishing the mesh vertex connection relationship. Experiments on multiple datasets show that the target reconstruction results of our method outperform those of other classical methods, and the relative accuracy of root-mean-square error for targets such as buildings, planes, and ships can reach up to 3 m or less. The accuracy of the earth mover's distance is better than 0.5.

Keywords

• Deformation inference
• satellite stereo images (SSIs)
• self-similar convolution
• three-dimensional (3-D) model extraction