Coupled Spatial-Spectral Constrained Convolutional Fusion Network for Hyperspectral and Panchromatic images

Abstract

Target monitoring is an important subject in machine vision. Hyperspectral image (HSI) can effectively assist the target detection and recognition effect of traditional optical images because of its rich spectral information. However, limited by pixel mixing, the resolution of HSI is generally lower than that of optical image, which restricts the monitoring distance and accuracy. Therefore, a fusion method of HSI and panchromatic image (PAN) based on coupled spatial-spectral constrained convolution neural network is proposed in this paper to improve the spatial resolution of HSI and reduce the spectral distortion. Through this approach, the linear spectral mixing model and the spatial-spectral transformation constraint model are incorporated into the learning stage of the coupled convolutional neural network, aiming to make full use of the spatial-spectral information of HSI and PAN, and improve the spectral fidelity of fused images. Experiments on several groups of HSI and PAN data sets show that compared with some currently proposed HSI and PAN fusion methods, the proposed approach has better spectral fidelity and lower fusion errors, so as to improve the monitoring distance and accuracy of machine vision in engineering applications and expand the engineering application scenarios.