Accent for Visible and Infrared Registration (AVIR): Attention Block for Increasing Patch Matching Rate Through Edge Emphasis

Abstract

In this paper, we propose an efficient attention module for visible and thermal infrared (TIR) matching deep learning networks. This method judges the right or wrong of heterogeneous sensor matching through the proposed deep learning model and increases the matching rate through the attention module using the edge-utilizing structure. This paper contributes to three aspects: 1) The first aspect is Convolutional Neural Network (CNN) structure comparisons for heterogeneous sensor registration. We consider the matching network as a classification problem when stacked heterogeneous sensor data become input of a single CNN network. Therefore, this paper shows result that is related with not only the network designed for heterogeneous sensor matching, but also various deep learning networks used for classification. 2) the second is a consideration for efficient attention module. The experiments show the module can replace lots of convolution blocks and the results achieve more better performance. The attention module uses a $1\times $ k filter and a k $\times $ 1 filter to extract horizontal and vertical edges and convolves two paths using them. 3) The third is suitable deep learning model for aerial complex visible and TIR data registration. To compare the various methods, we describe the calibration process of aerial visible and TIR data obtained directly from a drone. By using the calibrated data, this paper presents an AVIR attention block-based architecture that shows optimal matching results with minimal addition of parameters.

Keywords

• Visible
• thermal infrared
• heterogeneous sensor matching
• aerial data matching
• deep learning
• attention module