Multi‐task image restoration network based on spatial aggregation attention and multi‐feature fusion

Abstract

Abstract The main purpose of image restoration is to recover high‐quality image content from degraded versions. However, current mainstream models tend to focus solely on spatial details or contextual semantics, resulting in poor repair effects. To address this issue, a multi‐task image repair network based on spatial aggregation attention and multi‐feature fusion (SAAM) is proposed. It utilizes the global semantic information from the low‐resolution subnetwork to guide the local feature extraction of the high‐resolution subnetwork, thereby preserving the overall image structure while enhancing local details. Additionally, to enhance the model's understanding and representation capabilities of images, the feature fusion mechanism (FFM) is designed to merge feature information from different levels. Finally, the spatial aggregation attention mechanism SAAM enhances the accuracy and quality of image restoration by weighting the importance of different regions in the image at multiple scales. The experimental results demonstrate that the proposed SAAM method outperforms similar approaches in image denoising, deraining and decracking tasks in peak signal‐to‐noise ratio, structural similarity and learned perceptual image patch similarity metrics. The model also exhibits promising performance in restoring real old photos and murals which demonstrates its generalizability.

Keywords

• image denoising
• image restoration