Are deeper reflectors better goal-setters? AI-empowered analytics of reflective writing in pharmaceutical education

Abstract

Reflection and goal-setting are interrelated processes in well-established educational theories to promote in-depth self-reflection and self-regulated learning. Prior studies have considered reflection to be an important antecedent for meaningful goal-setting. Yet, there lacks empirical evidence to shed light on how students' abilities to reflect inform their abilities to set goals. Hence, in the present study, we aimed to quantify the connection between students' retrospective reflection and their subsequent goal-setting, and derive more in-depth insights to benefit educators in their teaching to promote deeper reflection, more specific goal-setting and better self-regulation. To this end, we utilised two fine-grained coding schemes, adapted from well-established reflection and goal-setting theories, respectively, as well as pertinent prior studies, to annotate the reflective and goal-setting elements within 600 student responses in pharmacy curricula. We visualised such elements as a network graph to study students' joint behavioural patterns in reflecting and setting goals. Then, we statistically analysed the correlation between students' reflective levels and the goal specificities using a Mann Whitney U test. We found that (1) descriptive reflection and goals that included content and actions with additional details more commonly presented jointly; (2) students who reflected deeply tended to set more specific goals. These findings are further summarised and discussed to guide educators to adopt reflective and goal-setting practices when designing teaching activities. Moreover, driven by these findings, we emphasised the significance of aiding instructors to provide timely assessment to students' written reflections so as to further ameliorate students' reflective abilities. Therefore, we attempted to automate such assessments using five traditional machine learning algorithms and one deep learning approach based on Bidirectional Encoder Representation of Transformers (BERT), and discovered that BERT gave the best performance in terms of identifying reflective sentences and differentiating various reflective elements.

Keywords

• Reflective writing
• Goal-setting
• Self-regulated learning
• Machine learning
• Deep learning
• BERT