Embodied Conversational Agents in Extended Reality: A Systematic Review

Abstract

Embodied conversational agents (ECAs) that can interact with users in a human-like manner have demonstrated promising potential in various endeavors. With the ongoing advancement in extended reality (XR) and artificial intelligence (AI), ECAs are becoming increasingly sophisticated. Although previous reviews have predominantly focused on ECAs for non-XR applications, a growing number of research papers are exploring the capabilities of ECAs that utilize XR technologies. However, no prior systematic review has focused explicitly on XR ECAs, leading to a gap in understanding how ECAs are designed, implemented, and evaluated within immersive environments. Our work identified the gap between the existing reviews and the current trends in XR ECAs. We began with 1,717 related papers from January 2014 to June 2024. We narrowed down the selection to 23 papers using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) framework, which employed an iterative screening procedure and criteria defined by our research team. The resulting papers were analyzed and discussed in terms of the features of the ECA application, its design and implementation, and use cases. Our analysis highlights key trends in XR ECA design, including the dominance of VR-based implementations using head-mounted displays, the prevalence of human-like and female-presenting agents, the move from rule-based to neural-based conversational systems, and the primary use cases in training, therapy, and social interaction. We also summarize the evaluation methods employed across studies and discuss future research directions for developing more adaptive and human-like ECAs in XR environments.

Keywords

• Embodied conversational agents
• extended reality
• conversational interaction
• use cases
• evaluation methods