The implementation of the cognitive theory of multimedia learning in the design and evaluation of an AI educational video assistant utilizing large language models

Abstract

The integration of Artificial Intelligence (AI) holds immense potential for revolutionizing education; especially, in contexts where multimodal learning experiences are designed. This paper investigated the potential benefits of Generative Artificial Intelligence (AI) in education, concentrating on the design and evaluation of an AI Educational Video Assistant tailored for multimodal learning experiences. The tool, utilizing the principles of the Cognitive Theory of Multimedia Learning (CTML), comprises three modules: Transcription, Engagement, and Reinforcement, each focusing on distinct aspects of the learning process. It Integraties Automatic Speech Recognition (ASR) using OpenAI's Whisper and Google's Large Language Model (LLM) Bard. Our twofold objective includes both the development of this AI assistant tool and the assessment of its effect on improving the learning experiences. For the evaluation, a mixed methods approach was adopted, combining human evaluation by nine educational experts with automatic metrics. Participants provided their perceptions on the tool's effectiveness in terms of engagement, content organization, clarity, and usability. Additionally, automatic metrics including Content Distinctiveness and Readability scores were computed. The results from the human evaluation suggest positive impacts across all assessed domains. The automatic metrics further proved the tool's ability in content generation and readability. Collectively, these preliminary results highlight the tool's potential to revolutionize educational design and provide personalized and engaging learning experiences.

Keywords

• Large language models
• Cognitive theory of multimedia learning
• Educational video
• ASR
• Google's bard