An Experiment of AI-Based Assessment: Perspectives of Learning Preferences, Benefits, Intention, Technology Affinity, and Trust

Abstract

The rising integration of AI-driven assessment in education holds promise, yet it is crucial to evaluate the correlation between trust in general AI tools, AI-based scoring systems, and future behavioral intention toward using these technologies. This study explores students’ perspectives on AI-assisted assessment in higher education. We constructed a comprehensive questionnaire supported by relevant studies. Several hypotheses grounded in the literature review were formulated. In an experimental setup, the students were tasked to read a designated chapter of a paper, answer an essay question about this chapter, and then have their answers evaluated by an AI-based essay grading tool. A comprehensive data analysis using Bayesian regression was carried out to test several hypotheses. The study finds that remote learners are more inclined to use AI-based educational tools. The students who believe that AI-based essay grading is less effective than teacher feedback have less trust in AI-based essay grading, whereas those who find it more effective perceive more benefit from it. In addition, students’ affinity for technology does not significantly impact trust or perceived benefits in AI-based essay grading.

Keywords

• artificial intelligence
• AI-based assessment
• technology affinity
• trust
• higher education
• EdTech