Identifying the Optimal Features in Multimodal Deception Detection

Abstract

Facial thermal imaging is a non-contact technology which can be useful for ubiquitous deceptive anxiety recognition. To date, studies investigating this technology have produced equivocal results in classification accuracy and finding the most correlated regions on the face. This study was conducted using our dataset with 41 subjects using two different protocols and three modalities (thermal, GSR and PPG). We selected and tracked five regions of interest (ROI) on each facial thermal imprint including periorbital, forehead, cheek, perinasal and chin that were mostly used in previous papers. By employing six statistical features, four feature reduction techniques and three classifiers, we attempted to identify the ROIs which are mostly associated with activation of the sympathetic nervous system to increase the final classification accuracy rate. The results of linear classification models show significant improvement of classification accuracy by using ROC feature selection method. We achieved 90.1% and 74.7% accuracy rate for thermal features in mock crime and best friend scenarios, respectively. Our experimental results show that perinasal and cheek areas have greater discriminatory power in comparison with other ROIs on the face.

Keywords

• facial thermal imaging
• region of interest
• multimodal deception detection
• control question test
• feature selection
• mock crime