Hyperspectral Remote Sensing Imagery Generation From RGB Images Based on Joint Discrimination

Abstract

Spatial resolution and spectral resolution both play an important role in the recognition of objects in hyperspectral remote sensing. However, the imaging characteristics of hyperspectral images (HSIs) result in a mutually restrictive relationship between the spatial and spectral resolutions. Generative adversarial networks (GANs) have achieved significant success in image generation. The introduce of the discriminators plays a key role in improving the reality. In this article, we propose an RGB to multiband hyperspectral imagery (150 bands) generation method based on GAN (R2HGAN). The method solves the high ill-posed problem and introduces high spectral resolution into RGB images by learning from multiple scenes of HSI. In R2HGAN, we extend the adversarial learning from spatial to spectral dimensions and joint discrimination is designed to generate HSIs closer to the real ones, where two discriminators (the conditional $D$ and the spectral $D$) are put forward to supervise the spectral similarity and the conditional reality of the HSI jointly. In detail, the conditional discriminator comprehensively judges the quality of each area in the reconstructed HSI. At the same time, to ensure that the generated spectra are close to the real ones, a spectral discriminator based on multilayer perceptron is designed. Through the experiments on GF-5 imagery, the method has significantly improved the quality of the generated images over other state-of-the-art methods.

Keywords

• Generation adversarial network (GAN)
• hyperspectral image (HSI)
• remote sensing
• spectral superresolution (SSR)