Equity at the forefront: A systematic research and development process of chatbot curriculum for massive open online courses

Abstract

This paper describes our equity-driven development of a chatbot curriculum for a journalism professional development center’s massive open online courses (MOOCs). A curriculum refers to entire student experiences that occur during the whole learning process in this context. We used a sequential and iterative development process with four phases to research and develop the chatbot curriculum: experimentation, evaluation, research and development reiteration, and implementation and evaluation in an ongoing course. In Phase 1, we tested whether a chatbot interface designed to answer frequently asked questions (FAQs) could provide more equitable student support than traditional FAQ pages, considering the widespread student frustration about FAQ pages. A total of 120 diverse students participated in this phase. In Phase 2, we conducted a case study to contrast the experiences of 27 non-native English users who used the FAQ chatbot, identifying what distinguished their experiences from those of 15 native English user counterparts. Based on this phase’s findings, the chatbot was further trained using an enhanced training set to better support non-native English users. In Phase 3, using a topic analysis result of 3,645 forum discussion posts from the center’s previous 21 courses, we ensured the chatbot’s expertise to cover nine additional topic themes. Phase 4 is where we examine students’ experiences with the chatbot’s latest version as a built-in course tool in MOOCs. This study suggests that systematic research and development of a chatbot curriculum is indispensable to fostering equitable learning environments in MOOCs.

Keywords

• chatbot curriculum
• design-based research
• equity
• mooc
• nlp chatbot