Single image reflection removal through multi‐scale gradient refinement

Abstract

Abstract Removing the undesired reflection layer from images taken through glass windows is an important yet challenging task. Many existing CNN‐based methods try to utilize the gradient as an important clue to guide the training and achieve better separation. But the scene depth of real‐world scenarios is usually uncontrollable, leading to the uncertainty of smooth level in the transmission and reflection layers, which makes it a great challenge to model the two layers in the gradient domain. This paper proposes a multi‐scale gradient refinement network to resolve this problem. First, it is suggested that even the two layers are usually partially smooth, their gradients can still be sharp in the down‐scaled samples. To this end, the separation is conducted at four different scales by minimizing the similarity of the two layers to boost the gradient sharpness prior. Second, it is considered that the separation performance of downscaled samples is usually superior to that of the high‐resolution images because of the sharper edges. For this reason, a cascade architecture is designed that takes the down‐scaled predictions to promote the high‐resolution decomposition stage‐by‐stage to recover the full‐resolution results. Besides, the scale‐wise memory mechanism is introduced into the prediction network to resolve the detail loss issue caused by the multi‐stage upscaling refinement process. The experimental results on benchmark datasets indicate that the new model surpasses several state‐of‐the‐art methods.