End to End Infrared and Visible Image Fusion With Texture Details and Contrast Information

Abstract

Infrared and visible image fusion combine data information from different sensors to achieve a richer description of the same scene. In order to highlight the salient features of the infrared image and the visible image in the fusion image and obtain a fusion image with good performance, an end-to-end infrared and visible image fusion algorithm is proposed in this paper. The contrast attention module and visible image cascade part are introduced in the generator, so that the fusion image can focus on the detail information in the visible image and the contrast information in the infrared image. And in order to retain more structural contour information in the original image, the contour loss is added to the content loss function. In addition, the contrast and detail information in infrared and visible images are balanced by two discriminators. And a goal-guided reward function is introduced into the discriminator, which further facilitates the generator to produce effective fused images. Finally, extensive fusion experiments on public datasets verify the advantages of the proposed algorithm compared with other classical algorithms, and ablation experiments demonstrate the effectiveness of the improved part of the algorithm.

Keywords

• End-to-end infrared and visible image fusion
• contrast attention module
• contour loss
• target-guided reward function