Bilateral Video Object Segmentation Using Dynamic Appearance Modeling and Higher-Order Potential

Abstract

Aiming at the problems of poor quality and low time efficiency of video object segmentation in complex scenes, this paper proposes a novel bilateral video object segmentation using dynamic appearance modeling and higher-order potential, which is formulated the video object segmentation problem as binary labeling of Markov random field (MRF) based on bilateral grid cells. Firstly, this paper resamples each pixel in the video sequence with labeled keyframes into a higher-dimensional bilateral grid, which greatly reduces the video data to be processed. Secondly, this paper constructs a graph-cut optimization model using non-empty grid cells as the nodes of graph. The key is to construct the dynamic appearance model with confidence measurements, and to introduce a robust higher-order potential into the energy function. Finally, this paper uses the max-flow/min-cut algorithm to solve the global optimization problem, and binary label assignment of each pixel is achieved to obtain the high-quality video object segmentation. The experimental results on DAVIS 2016 and SegTrack v2 datasets show that with less user interaction, this method can not only obtain high-quality video object segmentation results for videos with complex scenes, but also significantly improve the time efficiency of video object segmentation.

Keywords

• video object segmentation
• bilateral space
• bilateral grid
• confidence-based dynamic appearance model
• higher-order energy potential