Multimodal Siamese networks for dementia detection from speech in women

Abstract

Abstract The critical need for early and precise detection of dementia, a crippling cognitive illness that primarily affects women, is addressed by this study. Global healthcare systems face enormous hurdles as dementia becomes more common. The need for non-invasive and effective alternatives is highlighted because current diagnostic techniques are frequently invasive, expensive, and imprecise. To address this issue, our work presents a unique method for female dementia identification from speech using multimodal Siamese networks. In contrast to earlier models, our approach uses both transcript and audio data, utilizing the complementary information present in both modalities. Improving dementia detection accuracy and reliability is the main driving force for this study, particularly in the early stages when intervention can be more successful. Additionally, the information used in this study includes 104 people in the control group, 208 people with a dementia diagnosis, and 85 whose diagnosis is uncertain. There are 238 control files and 298 dementia files in the audio dataset, and 243 control files and 306 dementia files in the transcript database. This extensive dataset makes it possible to evaluate our suggested model with confidence. Moreover, multimodal Siamese networks—a cutting-edge technique that captures relationships between multimodal data—are a part of the basic methodology used. Our model has a greater accuracy of 99% on the Dementia Bank Database, demonstrating considerable improvements over earlier approaches. The assessment parameters, encompassing an Area Under the Curve (AUC) of 0.99, bolster the efficacy of our methodology. This paper will improve at-risk individuals’ quality of life by developing non-intrusive dementia detection for early diagnosis and intervention.

Keywords

• Dementia detection
• Speech in women
• Multimodal Siamese network
• Audio-transcript data