Computers and Education: Artificial Intelligence (Dec 2024)
AI in essay-based assessment: Student adoption, usage, and performance
Abstract
The rise of generative artificial intelligence (AI) has sparked debate in education about whether to ban AI tools for assessments. This study explores the adoption and impact of AI tools on an undergraduate research proposal assignment using a mixed-methods approach. From a sample of 187 students, 69 completed a survey, with 46 (67%) reporting the use of AI tools. AI-using students were significantly more likely to be higher-performing, with a pre-semester average GPA of 5.46 compared to 4.92 for non-users (7-point scale, p = .025). Most students used AI assistance for the highest-weighted components of the task, such as the research topic and methods section, using AI primarily for generating research ideas and gathering feedback. Regression analysis suggests that there was no statistically significant effect of AI use on student performance in the task, with the preferred regression specification estimating an effect size of less than 1 mark out of 100. The qualitative analysis identified six main themes of AI usage: idea generation, writing assistance, literature search, grammar checking, statistical analysis, and overall learning impact. These findings indicate that while AI tools are widely adopted, their impact on academic performance is neutral, suggesting a potential for integration into educational practices without compromising academic integrity.
