Two-Stream Edge-Aware Network for Infrared and Visible Image Fusion With Multi-Level Wavelet Decomposition

Abstract

Infrared and visible image fusion (IVIF) aims to generate a fused image with both salient target and rich textures from two different complementary modality images. To better integrate valuable edge information into the fused image, we first propose a novel two-stream network based on Auto-Encoder (AE) framework, which extracts deep hierarchical detail information at coarse scale from base stream by multi-level wavelet decomposition progressively and incorporates them into detail stream for information compensation. The aggregation of edge information ranging from coarse to fine facilitates a more comprehensive representation of contours and textures. Then, we propose a new feature fusion strategy, termed as Structural Feature Map Decomposition (SFMD). The first step is to decompose local patches of feature map with each modality into three independent components by Structural Patch Decomposition (SPD). In the second step, appropriate fusion rules are carefully designed for each component and the fused patch can be derived by inverse SPD. Our extensive experiments on several benchmark datasets show that our method outperforms seven compared state-of-the-art methods, especially in human visual perception.

Keywords

• Image fusion
• wavelet decomposition
• edge information
• multi-scale analysis