DGFusion: A Novel Infrared and Visible Image Fusion Method Based on Diffusion and Generative Adversarial Networks

Abstract

In current deep learning-based infrared and visible image fusion algorithms, the image processing step involves converting the RGB channels of visible image into luminance channels. These methods usually pay more attention to the texture details in the image and neglect the equally important color information, which contradicts human vision. Color information, a crucial role in human visual perception, is one of the most intuitive evaluation metrics for image fusion. In order to restore the color of fused images, researchers have made many attempts, such as enhancing brightness or contrast. but the fusion results are not satisfied. Dif-Fusion compensates for the lack of color information by creating a multi-channel data distribution. However, the balance of the multi-channel data distribution still poses a problem. Based on Dif-Fusion, we propose an enhanced algorithm named DGFusion. Firstly, we change the Information input mechanism to balance the weights of infrared image features and visible image, which can enhance the expression of infrared information. Meanwhile, for obtain deep-level features, UNet++ replaces the original U-Net structure of the diffusion model. Furthermore, we introduce a discriminator in the fusion network for superior texture detail preservation. We conducted comparative experiments and ablation studies, which shows that the DGFusion yields superior fusion results. Ablation experiments show that DGFusion improves on most metrics compared to the unmodified method, validating the effectiveness of our approach. Comparison experiments show that our method outperforms several state-of-the-art fusion methods in terms of metrics and visual effects.

Keywords

• Infrared and visible image fusion
• diffusion network
• UNet++
• generative adversarial network