A comprehensive survey: Image deraining and stereo‐matching task‐driven performance analysis

Abstract

Abstract Deraining has been attracting a lot of attention from researchers, and various methods have been proposed, especially deep‐networks are widely adopted in recent years. Their structures and learning become more and more complicated and diverse, making it difficult to analyze the contributions and improvements. In this paper, a comprehensive review for current rain removal methods is first provided to show their contributions. Specifically, they are reviewed in terms of handing rain streaks and rain mist. Second, besides evaluating their rain removal ability, they are also evaluated in terms of their impact on subsequent stereo‐matching task. To this end, a new deraining dataset is first prepared, called Rain‐Kitti2012 and Rain‐Kitti2015. They are created by adding rain part to clean image‐pairs in Kitti2012 and Kitti2015. By then, nine state‐of‐the‐art deraining methods are evaluated with full‐reference and no‐reference image quality assessment metrics. Furthermore, the blurriness and distortion types introduced during deraining are measured. Finally, three learning‐based stereo matching methods are compared, and they take the outputs of deraining methods as inputs. It is further discussed how derained images influence the accuracy of stereo matching, which can provide some insight for jointly handling rain removal and stereo matching. 1: A comprehensive review for the current rain removal methods is provided. They are categorized into rain‐streak‐oriented and rain‐mist‐oriented approaches in terms of degradation type, and are categorized into model‐driven and data‐driven approaches in terms of methodology. 2: A new image deraining dataset is introduced, which is the first dataset that can be used to perform stereo‐matching‐driven evaluation for deraining methods. The dataset is created by adding rain part to clean images in KITTI2012 and KITTI2015. 3: We evaluate 9 deep learning based deraining methods with full‐reference and no‐ reference metrics. In addition, the types of distortions produced by these methods are discussed and measured quantitatively. And, the impact of 9 deraining methods on the subsequent stereo matching task is evaluated, which can provide some insight on how to design stereo matching task‐driven deraining methods.

Keywords

• Optical, image and video signal processing
• Image recognition
• Computer vision and image processing techniques
• Video signal processing