Dynamic Spectral Guided Spatial Sparse Transformer for Hyperspectral Image Reconstruction

Abstract

Hyperspectral image (HSI) reconstruction plays a crucial role in compressive spectral imaging with coded aperture snapshot spectrometry. Although HSI reconstruction has attracted much attention in recent years, it remains a challenging problem. Existing deep learning-based methods leverage all the spectral information to reconstruct the HSI images without considering the spectral redundancy of HSI images, leading to high computational costs. In this article, we present an efficient method named dynamic spectral guided spatial sparse transformer (DGST). Specifically, DGST consists of three core modules as follows. 1) spectral sparse multihead self-attention hybrid spatial feature enhancement (SSHE) module, which employs a top-k spectral sparsity method to filter noise and redundant spectral information while extracting spectral information from HSI. 2) Spatial information compensation module, which utilizes a multiscale approach to extract spatial information and compensates for the spatial information neglected by SSHE. 3) Mask-guided spatial sparse multihead self-attention hybrid spectral enhancement module, which dynamically generates masks to guide the filtering of irrelevant regions, reducing computational costs while focusing on spatial information reconstruction. Our DGST improves the quality of HSI reconstruction by integrating spatial–spectral details and global information. Extensive experiments on public HSI reconstruction benchmark datasets demonstrate that our approach achieves state-of-the-art performance in end-to-end hyperspectral reconstruction. The superior performance of the proposed DGST is showcased on real and simulated hyperspectral imaging datasets.

Keywords

• Compressed imaging
• dual attention
• hyperspectral image (HSI) reconstruction
• sparse transformer