Self‐supervised depth completion with multi‐view geometric constraints

Abstract

Abstract Self‐supervised learning‐based depth completion is a cost‐effective way for 3D environment perception. However, it is also a challenging task because sparse depth may deactivate neural networks. In this paper, a novel Sparse‐Dense Depth Consistency Loss (SDDCL) is proposed to penalize not only the estimated depth map with sparse input points but also consecutive completed dense depth maps. Combined with the pose consistency loss, a new self‐supervised learning scheme is developed, using multi‐view geometric constraints, to achieve more accurate depth completion results. Moreover, to tackle the sparsity issue of input depth, a Quasi Dense Representations (QDR) module with triplet branches for spatial pyramid pooling is proposed to produce more dense feature maps. Extensive experimental results on VOID, NYUv2, and KITTI datasets show that the method outperforms state‐of‐the‐art self‐supervised depth completion methods.

Keywords

• image processing
• sensor fusion
• unsupervised learning