A Novel Approach to Enhancing Multi-Modal Facial Recognition: Integrating Convolutional Neural Networks, Principal Component Analysis, and Sequential Neural Networks

Abstract

Facial recognition technology is crucial for precise and rapid identity verification and security. This research delves into advancements in facial recognition technology for verification purposes, employing a combination of convolutional neural networks (CNNs), principal component analysis (PCA), and sequential neural networks. Unlike identification, our focus is on verifying an individual’s identity, that is a critical distinction in the context of security applications. Our goal is to enhance the efficacy and precision of face verification using several imaging modalities, including thermal, infrared, visible light, and a combination of visible and infrared. We use the pre-trained VGG16 model on the ImageNet dataset to extract features. Feature extraction is performed using the pre-trained VGG16 model on the ImageNet dataset, complemented by PCA for dimensionality reduction. We introduce a novel method, termed VGG16-PCA-NN, aimed at improving the precision of facial authentication. This method is validated using the Sejong Face Database, with a 70% training, 15% testing, and 15% validation split. While demonstrating a remarkable approaching 100% accuracy rate across visual and thermal modalities and a combined visible-infrared modality, it is crucial to note that these results are specific to our dataset and methodology. A comparison with existing approaches highlights the innovative aspect of our work, though variations in datasets and evaluation metrics necessitate cautious interpretation of comparative performance. Our study makes significant contributions to the biometrics and security fields by developing a robust and efficient facial authentication method. This method is designed to overcome challenges posed by environmental variations and physical obstructions, thereby enhancing reliability and performance in diverse conditions. The realised accuracy rates that the approach achieves across a variety of modalities demonstrate its promise for applications that use multi-modal data. This opens the door for the creation of biometric identification systems that are more trustworthy and secure. It is intended that the technology will be used in real-time settings for which the new modalities can be integrated in different situations.

Keywords

• Facial recognition (FR)
• convolutional neural networks (CNNs)
• principal component analysis (PCA)
• sequential neural networks (NNs)
• VGG16
• visible (VIS)