Video Frame Interpolation Based on Symmetric and Asymmetric Motions

Abstract

Video frame interpolation is the task to synthesize intermediate frames between consecutive frames to increase the frame rate. Recently, various deep-learning techniques have been proposed to interpolate intermediate frames more reliably. However, many existing methods use either symmetric (linear) or asymmetric (non-linear) schemes only to estimate motions for the warping process, resulting in unreliable interpolation results. In this paper, we propose a novel video frame interpolation network based on both symmetric and asymmetric motion-based warping modules, which can deal with linear and non-linear motions, as well as occlusions, effectively. The symmetric warping module estimates symmetric motions to generate intermediate frames, while the asymmetric one predicts asymmetric motions to address non-linear motions and occlusion problems. We combine symmetric and asymmetric warping results to reconstruct intermediate frames more reliably. We also develop the frame synthesis network to refine the combined warping results. Experimental results demonstrate that the proposed network outperforms state-of-the-art video interpolation algorithms and that the two types of warping modules work effectively in a complementary manner on various benchmark datasets.

Keywords

• Video frame interpolation
• convolutional neural network
• symmetric motion
• asymmetric motion
• motion estimation
• frame synthesis