U-Net-Based Multispectral Image Generation From an RGB Image

Abstract

Multispectral images have lower spatial resolution than RGB images. It is difficult to obtain multispectral images with both high spatial resolution and high spectral resolution because of expensive capture setup and sophisticated acquisition processes. In this paper, we propose a deep neural network structure based on U-Net to convert ordinary RGB images into multispectral images with high spectral resolution. Our variant U-Net neural network structure not only preserves detailed features of RGB images, but also promotes the fusion of different feature scales, enhancing the quality of multispectral image generation. Apart from the training stage, our proposed method does not require low-resolution multispectral images, as do some earlier learning-based methods; multispectral images can be obtained using only the corresponding RGB high-resolution images. We also employ the Inception block to achieve richer image features and the feature loss function to optimize the non-local features. Our proposed algorithm achieves state-of-the-art visual effects and quantitative measurements such as RMSE and rRMSE on several different public datasets.

Keywords

• Convolutional neural networks
• multispectral image
• U-Net