Facilitating university admission using a chatbot based on large language models with retrieval-augmented generation

Abstract

University admission consultation is a professional service that assists students with the university application process. Typically, accessing this service entails exploring university websites, directly contacting faculty members and officers via phone calls or emails, and engaging educational intermediaries. University admission consultation is crucial for both students and institutions. However, conventional consultation methods face challenges such as time and spatial constraints, leading to a growing interest in utilizing chatbots for university admission consultation. In this study, we propose a novel approach that leverages generative pretrained transformer (ChatGPT 3.5) models and implements the retrieval-augmented generation technique using the LlamaIndex framework. To evaluate the effectiveness of this approach, we applied it to undergraduate admission data from three universities: a science and technology university in the United States, a comprehensive university in Kenya, and a comprehensive university in Hong Kong. We also gathered feedback from 53 high school students who tested the chatbot. The results demonstrated a significant improvement in average accuracy, from 41.4% with the ChatGPT 3.5 model to 89.5% with the proposed chatbot, with peak accuracy reaching 94.7%. User reviews also indicated a generally positive perception of the admission chatbot. This methodology has the potential to revolutionize university admissions by utilizing chatbots based on large language models with retrieval-augmented generation.

Keywords

• university admissions
• chatbot
• gpt
• large language models
• retrieval-augmented generation