PeerJ Computer Science (Jun 2024)

Deep learning-based dimensional emotion recognition for conversational agent-based cognitive behavioral therapy

  • Julian Striegl,
  • Jordan Wenzel Richter,
  • Leoni Grossmann,
  • Björn Bråstad,
  • Marie Gotthardt,
  • Christian Rück,
  • John Wallert,
  • Claudia Loitsch

DOI
https://doi.org/10.7717/peerj-cs.2104
Journal volume & issue
Vol. 10
p. e2104

Abstract

Read online Read online

Internet-based cognitive behavioral therapy (iCBT) offers a scalable, cost-effective, accessible, and low-threshold form of psychotherapy. Recent advancements explored the use of conversational agents such as chatbots and voice assistants to enhance the delivery of iCBT. These agents can deliver iCBT-based exercises, recognize and track emotional states, assess therapy progress, convey empathy, and potentially predict long-term therapy outcome. However, existing systems predominantly utilize categorical approaches for emotional modeling, which can oversimplify the complexity of human emotional states. To address this, we developed a transformer-based model for dimensional text-based emotion recognition, fine-tuned with a novel, comprehensive dimensional emotion dataset comprising 75,503 samples. This model significantly outperforms existing state-of-the-art models in detecting the dimensions of valence, arousal, and dominance, achieving a Pearson correlation coefficient of r = 0.90, r = 0.77, and r = 0.64, respectively. Furthermore, a feasibility study involving 20 participants confirmed the model’s technical effectiveness and its usability, acceptance, and empathic understanding in a conversational agent-based iCBT setting, marking a substantial improvement in personalized and effective therapy experiences.

Keywords