Real-Time Stereo Vision Hardware Accelerator: Fusion of SAD and Adaptive Census Algorithm

Abstract

Stereo vision technology, as a significant branch of computer vision, has been widely applied in fields such as robot navigation, autonomous driving, and 3D reconstruction. Achieving embodied intelligence through edge hardware platforms remains challenging in balancing power consumption, real-time performance, and accuracy. Although the semi-global stereo matching algorithm has proven effective in balancing disparity map accuracy and computational complexity, its matching accuracy is often limited by weak textures, disparity discontinuities, lighting variations, and noise. To address the limitations of existing semi-global stereo matching algorithms and the power constraints of hardware platforms, this paper proposes a stereo matching algorithm based on FPGA that integrates the Sum of Absolute Differences (SAD) with adaptive Census transform. The goal is to enhance image and edge information and design a compact and efficient stereo vision hardware accelerator. During the hardware implementation phase, a pixel-level pipelined parallel matching cost computation structure is proposed. This structure significantly reduces data buffer requirements through multi-step parallel computation. Additionally, a two-stage four-layer parallel pipelined semi-global cost aggregation architecture is adopted, which effectively balances hardware resource utilization while maintaining accuracy. Evaluations of the Middlebury dataset show that compared to traditional SAD and Census algorithms, the proposed algorithm improves matching accuracy by 15.67% and 15.1%, respectively. On the Xilinx Zynq-7 platform, for images with resolutions of $1280\times 720$ and $640\times 480$ , the processing speeds reach 54.24fps and 81.34fps, respectively.

Keywords

• Stereo vision
• FPGA
• real-time
• semi-global matching
• adaptive window