Robust graph fusion and recognition framework for fingerprint and finger‐vein

Abstract

Abstract The human finger is the essential carrier of biometric features. The finger itself contains multi‐modal traits, including fingerprint and finger‐vein, which provides convenience and practicality for finger bi‐modal fusion recognition. The scale inconsistency and feature space mismatch of finger bi‐modal images are important reasons for the fusion effect. The feature extraction method based on graph structure can well solve the problem of feature space mismatch for the finger bi‐modalities, and the end‐to‐end fusion recognition can be realised based on graph convolutional neural networks (GCNs). However, this fusion recognition strategy based on GCNs still has two urgent problems: first, lack of stable and efficient graph fusion method; second, over‐smoothing problem of GCNs will lead to the degradation of recognition performance. A novel fusion method is proposed to integrate the graph features of fingerprint (FP) and finger‐vein (FV). Furthermore, we analyse the inner relationship between the information transmission process and the over‐smoothing problem in GCNs from an optimisation perspective, and point out that the differentiated information between neighbouring nodes decreases as the number of layers increases, which is the direct reason for the over‐smoothing problem. A modified deep graph convolution neural network is proposed, aiming to alleviate the over‐smoothing problem. The intuition is that the differentiated features of the nodes should be properly preserved to ensure the uniqueness of the nodes themselves. Thus, a constraint term to the objective function of the GCN is added to emphasise the differentiation features of the nodes themselves. The experimental results show that the proposed fusion method can achieve more satisfied performance in finger bi‐modal biometric recognition, and the proposed constrained GCN can well alleviate the problem of over‐smoothing.