Axial Constraints for Global Matching-Based Optical Flow Estimation

Abstract

Optical flow estimation is a fundamental task that aims to find the 2-dimensional motion field by identifying correspondences between two input images. For quite a long time, the correlation volume followed by convolutional neural networks (CNN) to directly estimates the optical flow was a predominant pipeline. However, several pioneering methods proposed global matching recently, pointing out the limitation that CNN-based methods are struggling to handle large displacements due to their locality. Global matching is the step that identifies global correspondences at the pixel-level using entire correlation volumes at once with simple operations like softmax. However, when global matching with softmax is combined with commonly used regression loss in optical flow estimation, there will be a vast number of possible correlation volumes that can minimize the regression loss and correctly estimate correspondences. In other words, the training objective induces a one-to-many solution problem resulting in the presence of noisy gradients. In this paper, the necessity for more constraints on the correlation volume to mitigate the aforementioned ill-posed problem is discussed. To acquire such constraints, axial cross-entropy loss (i.e. axial constraints) to restrict the correlation volume to have low variance with designed pseudo ground truth is proposed. Experimental results show that axial constraints are applicable to off-the-shelves global matching-based optical flow estimation frameworks easily and lead to both quantitative and qualitative performance improvement without any architectural changes.

Keywords

• Global matching
• optical flow estimation
• under-constrained problem