Real-time Binocular Depth Estimation Algorithm Based on Semantic Edge Drive

Abstract

Aiming at the problem of ill-posed regions with blurred disparity edges,unsmooth disparity,discontinuous disparity of a single object,and holes in stereo matching,a lightweight real-time binocular depth estimation algorithm is proposed,which uses the semantic tags obtained by semantic segmentation of the scene graph and the edge detail images obtained by edge detection asauxi-liary loss,and the ground truth image as the main loss,to construct the joint loss function which can better supervise the generation of the disparity map.In addition,a lightweight feature extraction module is constructed to reduce the redundancy of the feature extraction stage,which can better simplify the feature extraction steps,and improve the real-time and lightness of the network.Finally,the idea of from coarse to fine is used to realize the gradual refinement process of the disparity map with fusion of low-resolution disparity map deformation and high-resolution feature map to generate disparity maps of different scales in stages,meanwhile,the detailed features are gradually enriched,thus obtaining the final accurate disparity map.The 3px error rate of 1.72% is obtained on the KITTI 2012 dataset,the Vintge error rate on the Middlebury 2014 dataset is 1.23%,the Playroom error rate is 2.23%,and the Recycle error rate is 1.65%.Meanwhile,the calculation time on the Scene Flow dataset reaches 0.76 s with 2.4 G memory occupation,which significantly improves the accuracy and computational efficiency of stereo matching algorithms in the ill-posed regions,meets the real-time requirements in engineering practice,and has important guiding significance for real-time 3D reconstruction tasks.

Keywords

• stereo matching|semantic understanding|edge extraction|end-to-end network|from coarse to fine