Real-Time Stereo Matching Network Based on 3D Channel and Disparity Attention for Edge Devices Toward Autonomous Driving

Abstract

Stereo matching is an important component technology that constitutes the 3D perception capability of autonomous vehicles. On resource-constrained edge devices, it is very important to compute in real-time with very low time. However, most stereo matching networks focus on generating disparity maps on high-end GPUs, which do not meet the real-time requirements on edge devices. To solve this problem, a new stereo matching network is proposed in this paper to achieve real-time stereo matching on edge devices. The proposed network greatly improves the inference speed by constructing a low-resolution feature extractor, and by using multi-stage residual methods for stereo matching. In particular, we propose a method that combines the group-wise L1 distance & the group-wise correlation cost volume and an effective attention-based 3D cost aggregation method. Our network achieves a good balance between speed and accuracy on the KITTI 2012 and KITTI 2015 datasets. The proposed network achieves 2.77% and 3.44% accuracy (D1-all) on KITTI 2012 and KITTI 2015, respectively. With TensorRT, the proposed network achieves 31.8 FPS and outperforms the real-time results of most state-of-the-art networks on NVIDIA Jetson Nano edge devices.

Keywords

• Real-time stereo matching
• autonomous driving
• edge device
• group-wise L1 distance & group-wise correlation
• attention-based 3D cost aggregation