MiEmo: A multi-modal platform on emotion recognition for children with autism spectrum condition

Abstract

Introduction: This study presents MiEmo, a multi-modal digital platform designed to improve emotion recognition in children with Autism Spectrum Condition (ASC). The platform integrates serious games with music and color as feedback mechanisms to strengthen emotion understanding in addition to traditional visual interventions such as pictures and videos. The study aims to assess the usability and potential effectiveness of MiEmo in supporting therapy for children with medium- and high-functioning ASC. Methods: A pilot usability study was conducted in two rehabilitation centers involving 19 children, 8 with medium and 11 with high functioning ASC. Participants engaged with six training activities, or exergames, on the MiEmo platform that implemented multi-modal feedback (music pieces and colored animations associated to the emotion). The System Usability Scale (SUS) and qualitative feedback from therapists were used to evaluate the platform's usability. Results: The average SUS scores were 86.88 for children with medium-functioning ASC and 96.75 for those with high-functioning ASC, indicating positive usability. Therapists noted that while the platform was well-received, further updates are needed for better adaptation to medium-functioning children. Multi-modal feedback, particularly music and color, was found to enhance emotion recognition, with children responding well to the integration of these sensory cues. Conclusion: The study demonstrates that MiEmo has significant potential as a tool for socio-emotional training, particularly for high-functioning children. However, limitations such as the small sample size, short intervention duration, and lack of a control group suggest that future studies with larger participant groups are necessary to validate these findings and assess long-term effects.

Keywords

• ASC
• Autism intervention
• Emotion recognition
• Serious game
• Exergame
• Social skills