DIVIDUAL: A Disentangled Visible and Infrared Image Fusion Contrastive Learning Method

Abstract

Visible and Infrared Image Fusion (VIIF), as a vital fundamental component in vehicle applications, has been attracting plenty of attention from the academic and industrial communities over past few years. Various deep learning based methods have been proposed to effectively fuse visible and infrared images for improving the comprehensiveness of vehicle sensing and monitoring capabilities. However, due to the complex coupling property of the vehicle observational environment, it is still a challenging problem to effectively decouple and fusion visible and infrared images. To address this problem, we propose a DIsentangled Visible and InfrareD image fUsion contrAstive Learning method (DIVIDUAL). For capturing common and complementary information between the two domains, DIVIDUAL contains a self-supervised decoupling framework to separate domain-invariant and domain-specific representations. Meanwhile, for removing the noise in domain-specific representations and extracting clean domain-invariant representations, DIVIDUAL deploys a decoupling contrastive loss to effectively separate noise information and retain critical information in domains. Finally, DIVIDUAL generates fused images in an end-to-end manner. Extensive performance and generalization experiments on TNO and RoadScene datasets demonstrate that DIVIDUAL has superior visual results.

Keywords

• disentangled learning
• image fusion
• contrastive learning
• self-unsupervised learning