Deep Unfolding Network for Multi-Band Images Synchronous Fusion

Abstract

This study proposes a new deep neural network to solve the multi-band image synchronous fusion problem (MBF-Net). Unlike other deep learning-based methods, our network architecture design combines the ideas of model-driven and data-driven methods, so it is more interpretable. First, a new multi-band image synchronous fusion model is proposed. The source image in the data fidelity terms and the prior regularization are implicitly represented by the deep learning network and jointly learned from the training data. The proposed model is then solved using a half quadratic splitting (HQS) algorithm and unfolded into a deep fusion network. In addition, a new saliency loss function is proposed to retain thermal radiation information to enhance the fusion effect. Finally, the experimental results on the TNO dataset demonstrated the effectiveness of the proposed MBF-Net.

Keywords

• Image fusion
• multi-band images
• total variation
• optimization model