Exploiting Sequence Analysis for Accurate Light-Field Depth Estimation

Abstract

Depth estimation for light field (LF) images is the cornerstone of many applications of light field cameras, such as 3D reconstruction, defects inspection, face liveness detection, and so forth. In recent years, convolutional neural network (CNN) has dominated the primary workhorse for depth estimation. However, the interpretability of the network and the accuracy of the depth estimation results still need to be improved. This paper uses the conditional random field (CRF) theory to explain and model the LF depth estimation. Further, from the perspective of sequence analysis, we extract the sequence features of epipolar plane image (EPI) patches with recurrent neural network (RNN) and serve as the unary term of the energy function in the CRF. Then, a unified neural network (called as LFRNN) is designed to solve the CRF and get the disparity map. Our LFRNN builds upon two-stage architecture, involving a local depth estimation and a depth refinement. In the first part, we design an RNN to analyze the vector sequences in EPI patches and obtain local disparity values. There are two thinking behind the design of this part. The first is the general principle that the slope of the straight line in the EPI is inversely proportional to the depth; the second is our unique observation that those straight lines are distributed in vector sequences. In the second part, continuous CRF is used to optimize the output of the first part. We train LFRNN on a synthetic LF dataset and test it on both synthetic and real-world LF datasets. Quantitative and qualitative results validate the superior performance of our LFRNN over the state-of-the-art methods.

Keywords

• Computer vision
• depth estimation
• light field imaging
• deep learning
• sequence analysis