Double Weight-Based SAR and Infrared Sensor Fusion for Automatic Ground Target Recognition with Deep Learning

Abstract

This paper presents a novel double weight-based synthetic aperture radar (SAR) and infrared (IR) sensor fusion method (DW-SIF) for automatic ground target recognition (ATR). IR-based ATR can provide accurate recognition because of its high image resolution but it is affected by the weather conditions. On the other hand, SAR-based ATR shows a low recognition rate due to the noisy low resolution but can provide consistent performance regardless of the weather conditions. The fusion of an active sensor (SAR) and a passive sensor (IR) can lead to upgraded performance. This paper proposes a doubly weighted neural network fusion scheme at the decision level. The first weight ( α ) can measure the offline sensor confidence per target category based on the classification rate for an evaluation set. The second weight ( β ) can measure the online sensor reliability based on the score distribution for a test target image. The LeNet architecture-based deep convolution network (14 layers) is used as an individual classifier. Doubly weighted sensor scores are fused by two types of fusion schemes, such as the sum-based linear fusion scheme ( α β -sum) and neural network-based nonlinear fusion scheme ( α β -NN). The experimental results confirmed the proposed linear fusion method ( α β -sum) to have the best performance among the linear fusion schemes available (SAR-CNN, IR-CNN, α -sum, β -sum, α β -sum, and Bayesian fusion). In addition, the proposed nonlinear fusion method ( α β -NN) showed superior target recognition performance to linear fusion on the OKTAL-SE-based synthetic database.

Keywords

• SAR
• IR
• fusion
• double weights
• linear
• nonlinear
• deep learning
• OKTAL-SE