Attention‐based video object segmentation algorithm

Abstract

Abstract To improve the segmentation performance on videos with large object motion or deformation, a novel scheme is proposed which has two branches. In one branch, the attention mechanism is first utilized to highlight objects‐related features. Then, to well consider the temporal coherence of videos, Conv3D is integrated to capture short‐term temporal features, and the designed attention residual convolutional long–short‐term memory is adopted to capture the long–short‐term temporal information of objects under the interference of redundant video frames. Meanwhile, considering the negative effect of background motion, in another branch, the optical flow‐based prediction model is introduced to predict objects regions in subsequent video frames with the annotated initial frame. At last, based on the fused results of two branches, the global thresholds and noising area clean method are employed to obtain segmented objects. The experiments on DAVIS2016 and CDnet2014 exhibit the competitive performance of the proposed scheme.

Keywords

• Optical, image and video signal processing
• Computer vision and image processing techniques
• Video signal processing
• Neural nets