Minds vs machines: A comparative study of AI and teacher-generated summaries in ELT

Abstract

This study investigates the differences between human-generated and AI-generated summaries in a remote English as a Foreign Language (EFL) lesson setting, addressing the research problem of how each approach captures and interprets lesson content. Utilizing Zoom-AI as the AI summarization tool, the study compares its output with summaries created by ten human educators. Each participant summarized the same lesson, providing a basis for direct comparison. The methodology involved qualitative analysis, focusing on aspects such as content comprehensiveness, pedagogical judgment, contextual understanding, and the recognition of classroom dynamics. The key findings have revealed that while the AI-generated summary is significantly more efficient in capturing the content, it lacks depth in educational insights and contextual nuances. Conversely, human-generated summaries appear to have provided richer educational judgments and a better understanding of classroom interactions but sometimes deviated from the core content, decreasing their educational value. The study suggests a complementary approach, integrating AI’s efficiency with human expertise through a human-in-theloop system to enhance the overall quality and utility of educational summaries. These results have important implications for integrating of AI in educational settings, highlighting the potential for AI to assist educators and the irreplaceable need for the nuanced understanding and contextual interpretation that human educators provide.

Keywords

• Artificial Intelligence (AI)
• Artificial Intelligence in Education (AIEd)
• Discourse Analysis
• Teacher Education
• Zoom
• English as a Foreign Language (EFL)