Integrating Bert With CNN and BiLSTM for Explainable Detection of Depression in Social Media Contents

Abstract

Depression is a prevalent mental health condition that significantly impacts individuals’ lives. Early detection of depression is crucial for timely intervention and improved outcomes. However, traditional machine learning approaches are constrained by the limited amount of annotated data and lack of model transparency. This study aims to address these challenges by leveraging social media data and advanced natural language processing techniques to develop effective and explainable models for depression detection. The study focuses on two main objectives. The first objective is to develop and evaluate fine-tuned Bidirectional Encoder Representations from Transformers (BERT), BERT with Bidirectional Long Short-Term Memory (BERT-BiLSTM), and BERT with Convolutional Neural Network (BERT-CNN) models, and compare their performance with MentalBERT, a state-of-the-art model for mental health detection. The second objective is to observe the key features used by the BERT models to make the decision-making using Transformer Interpretability Beyond Attention Visualization and Average Attention Weight methods. The study utilizes three publicly available datasets: the Depression Reddit Dataset, the Sentiment Analysis for Tweets Dataset, and the Mental Health Corpus. The results demonstrate that the proposed models, especially BERT-BiLSTM and BERT-CNN, achieve superior performance compared to MentalBERT, particularly regarding accuracy and F1-score. Notably, BERT-CNN achieved exceptional accuracy scores of 0.982, 0.961, and 1.0 on the Depression Reddit Dataset, the Mental Health Corpus, and the Sentiment Analysis for Tweets Dataset, respectively, demonstrating its robust performance across different social media contexts. The attention map visualizations provide valuable insights into the language patterns and key features associated with depression in social media posts. This study contributes to the mental health field by presenting novel and explainable models for depression detection using social media data. The proposed approaches have the potential to assist mental health professionals in early identification and intervention, ultimately improving the lives of individuals affected by depression.

Keywords

• Automatic depression detection
• BERT
• CNN
• BiLSTM
• MentalBERT
• deep learning