MSTNet: a multi-stage progressive network with local–global transformer fusion for image restoration

Abstract

Abstract Image restoration is a challenging and complex problem involving recovering the original clear image from a degraded or noisy image. In the medical field, image restoration techniques can significantly improve the quality of endoscopic images, helping doctors make more accurate diagnoses and providing higher-quality data support for computer vision-assisted detection. Existing methods for image restoration mainly use convolutional neural networks (CNNs) or Transformer models, which have different advantages and limitations in capturing spatial and channel information of the image. This paper proposes a novel Multi-Stage progressive image restoration Network based on a blend of local–global Transformers, named MSTNet. Our network consists of three stages, each using a different type of Transformer module to obtain local and global information. The first two stages use window-based Transformer modules, which can effectively extract local spatial information within each window. The third stage uses channel-level Transformer modules to capture global channel information across the whole image. We also introduce a fusion module to combine the features from different Transformer branches and obtain a comprehensive and accurate feature representation. We conduct extensive experiments on various image restoration tasks, such as deblurring and denoising, evaluating our approach on both general image restoration datasets and our proposed colon dataset. The results demonstrate the effectiveness and superiority of our network over state-of-the-art methods.

Keywords

• Image restoration
• Transformer
• Multi-stage network
• Feature fusion
• Colon dataset