Two‐stage learning framework for single image deraining

Abstract

Abstract Single image deraining methods have been extensively studied for its ability to remarkably improve the performance of computer vision tasks in rainy environments. However, most existing rain removal methods still have two major drawbacks which are hindering the technology development. First, the rain streaks are seriously coupled with the background information in a single rainy image, which leads to incorrect identification of rain streaks by many methods and further makes the loss of texture details in the rain removal results. Second, they spend excessive computational cost, which is not conducive to practical applications. To address these issues, a progressive separation network (PSN) is proposed by decomposing the rain removal task into two stages, the bilateral grid learning stage and the joint feature refinement stage, from a novel perspective. The bilateral grid learning stage is designed to expand the distance between the rain streaks and the background information while preserving the image edge details to guide the subsequent refinement. For the joint feature refinement stage, a dual‐path interaction module is constructed to dynamically and gradually decouple the rain streak content and the intermediate features of the clear image details. In addition, an activation‐free feature refinement block is designed to further improve the computational efficiency by removing or replacing the activation function without loss of accuracy. Extensive experiments on synthetic and real datasets show that PSN outperforms state‐of‐the‐art rain removal methods in terms of quantitative accuracy and subjective visual quality. Furthermore, competitive results are derived by extending PSN to the defogging task.

Keywords

• bilateral grid
• joint feature refinement
• progressive separation