An Edge-Enhanced Branch for Multi-Frame Motion Deblurring

Abstract

Non-uniform deblurring is one of the most important image restoration tasks for providing appropriate information for subsequent applications that require image recognition. Conventional deep learning-based multi-frame deblurring methods collectively handle many types of non-uniform blurring, such as camera shakes and motion blur. However, edge and high-frequency component restoration is still insufficient for severe motion blur. This paper proposes an auxiliary edge-enhanced branch to support motion blur restoration for deep learning-based multi-frame deblurring methods. The background region in an image with little motion generally has more edge information, whereas the moving object region lacks high-frequency components. Thus, we propose a motion orthogonal edge (MOE) feature that extracts only the edge information of moving objects by computing the pixel-wise inner product between the edge information obtained by Sobel filters and the optical flow representing motion in the image. MOEs can emphasize only the edges of moving objects excluding the backgrounds. In this paper, we add an edge-enhanced branch that computes MOEs to a conventional multi-frame deblurring method, the spatio-temporal deformable attention network, and call it ESTDANet. We introduce additional frequency reconstruction loss to restore high-frequency components and compare our proposed ESTDANet with the conventional baseline method in our comparative experiments. Furthermore, we introduce motion-weighted SSIM maps to distinguish the deblurring accuracy in motion regions spatially. The results show that our edge-enhanced branch aids edge restoration in the motion deblurring of conventional methods.

Keywords

• Motion blur
• deblurring
• edge enhancement
• image restoration
• optical flow
• SSIM