A novel factor graph-based optimization technique for stereo correspondence estimation

Abstract

Abstract Dense disparities among multiple views are essential for estimating the 3D architecture of a scene based on the geometrical relationship between the scene and the views or cameras. Scenes with larger extents of homogeneous textures, differing scene illumination among the multiple views and with occluding objects affect the accuracy of the estimated disparities. Markov random fields based methods for disparity estimation address these limitations using spatial dependencies among the observations and among the disparity estimates. These methods, however, are limited by spatially fixed and smaller neighborhood systems or cliques. Recent learning-based methods generate rich set of stereo features for generating cost volume and estimating disparity. In this work, we present a new factor graph-based probabilistic graphical model for disparity estimation that allows a larger and a spatially variable neighborhood structure determined based on the local scene characteristics. Our algorithm improves the accuracy of disparity estimates in stereo image pairs with varying texture and illumination characteristics by enforcing spatial dependencies among scene characteristics as well as among disparity estimates. We evaluated our method using the Middlebury benchmark stereo datasets and the Middlebury evaluation dataset version 3.0 and compared its performance with recent state-of-the-art disparity estimation algorithms. Our factor graph-based algorithm provided disparity estimates with higher accuracy when compared to the recent non-learning- and learning-based disparity estimation algorithms. The factor graph formulation can be used for obtaining maximum a posteriori estimates from models or optimization problems with complex dependency structure among hidden variables. The strategies of using a priori distributions with shorter support and spatial dependencies were useful for reducing the computational cost and improving message convergence in the model. The factor-graph algorithm is also useful for other dense estimation problems such as optical flow estimation.