Multifiltering MLP for Spectral Super-Resolution With Remote Sensing Image Verification

Abstract

Spectral super-resolution (SSR) has become an attractive approach to reconstructing hyperspectral images (HSIs) from more available RGB images or multispectral images owing to the powerful representation capability of deep learning. Prevailing efforts concentrate on reconstructing all channels of HSIs via constraining with groundtruth on the whole, ignoring precision of discriminative spectral characteristics (e.g., specific absorption peaks), which is critical for downstream tasks, such as fine-grained hyperspectral classification to recognize analogous ground objects with similar spectral characteristics. Therefore, we introduce an efficient multifiltering multilayer perception (MLP) for SSR (multi filtering MLP for spectral super resolution (MF-SSR)) to reconstruct meticulous and high-fidelity HSIs in this article, paving the roads toward downstream tasks based on recovered HSIs. A specific MF-MLP block is presented to individually reconstruct distinctive spectral-wise characteristics by repressing surrounding interferences from near channels. The flexible filtering ratios and positions in MF-MLP randomly disrupt the recovered main channel and surrounding channel range, which delineates specific absorption peaks to further represent the unique reflectivity or radiance of the light in each pixel. Besides, a cross spatio-spectral attention module is explicitly presented to simultaneously extract pixel-wise correlation and channel-wise affinity to amplify the consistency of the same substance and the diversities of different substances in a complementary mode. Comprehensive SSR experiments on four datasets and further classification verification based on reconstructed HSIs for Pavia University and GF-X datasets have demonstrated the superiority and practicability of our MF-SSR.

Keywords

• Distinctive characteristics
• hyperspectral classification
• multifiltering
• remote sensing
• spectral super-resolution (SSR)