Short-term learning effect of ChatGPT on pharmacy students' learning

Abstract

Introduction: Students in pharmacy are positive towards integrating artificial intelligence and ChatGPT into their practice. The aim of this study was to investigate the direct short-term learning effect of using Chat GPT by pharmacy students. Methods: This was an experimental randomized study. Students were allocated into two groups; the intervention group (n = 15) used all study tools and ChatGPT, while the control group (n = 16) used all study tools, except ChatGPT. Differences between groups was measured by how well they performed on a knowledge test before and after a short study period. Results: No significant difference was found between the intervention and control groups in level of competence in the pretest score (p = 0.28). There was also no significant effect of using ChatGPT, with a mean adjusted difference of 0.5 points on a 12-point scale. However there was a trend towards a higher proportion of ChatGPT participants having a large (at least four point) increase in score (4 out of 15) vs control group (1 out of 16). Conclusion: There is a potential for positive effects of ChatGPT on learning outcomes in pharmacy students, however the current study was underpowered to measure a statistically significant effect of ChatGPT on short term learning.

Keywords

• ChatGPT
• Large language modelling
• Pharmacy education
• Experimental study design