Taking All the Factors We Need: A Multimodal Depression Classification With Uncertainty Approximation

Abstract

Depression and anxiety are prevalent mental illnesses that are frequently disregarded as disorders. It is estimated that more than 5% of the population suffers from depression or anxiety. Although there have been a number of studies in these fields, the majority of the research focuses on one or two factors for detection purposes, whereas these factors are not mutually inclusive and vary among studies. To mitigate these issues, we first consider all possible symptoms associated with depression and develop a multimodal diagnosis system that may take into account any number of patient-specific factors. If multiple factors can be addressed within a single learning model, it is advantageous for data collection and future development. To facilitate training with missing modalities, we propose an attention-based multimodal classifier with selective dropout and normalization, which can facilitate the training of various multimodal datasets on one neural network. We have experimented with three multimodal datasets with varying modalities to show the impact of combined training in one neural network and achieved an F1 score of 0.945. However, missing modalities in the model can create uncertainty in the prediction. For uncertainty approximation, the Monte Carlo dropout (MC dropout) and the spectral-normalized neural Gaussian process (SNGP) with the coefficient of variation and S1-Score metrics are implemented to provide important information about multimodal diagnosis processes. In the experiment, selective dropout with SNGP achieved a coefficient of variation in loss of 0.384 and an S1-score of 0.9374.

Keywords

• Deep learning
• multi-modal neural network
• uncertainty approximation
• ensemble