BMC Psychology (Sep 2024)

Validating the ChatGPT Usage Scale: psychometric properties and factor structures among postgraduate students

  • Mohamed Nemt-allah,
  • Waleed Khalifa,
  • Mahmoud Badawy,
  • Yasser Elbably,
  • Ashraf Ibrahim

DOI
https://doi.org/10.1186/s40359-024-01983-4
Journal volume & issue
Vol. 12, no. 1
pp. 1 – 7

Abstract

Read online

Abstract Background The rapid adoption of ChatGPT in academic settings has raised concerns about its impact on learning, research, and academic integrity. This study aimed to develop and validate a comprehensive ChatGPT Usage Scale specifically tailored to postgraduate students, addressing the need for a psychometrically sound instrument to assess the multidimensional nature of ChatGPT usage in higher education. Methods A cross-sectional survey design was employed, involving 443 postgraduate students from two Egyptian universities. The initial 39-item scale underwent Exploratory Factor Analysis (EFA) using principal component analysis with Varimax rotation. Confirmatory Factor Analysis (CFA) was conducted to assess the model fit and psychometric properties of the final 15-item measure. Internal consistency reliability was evaluated using Cronbach’s alpha and McDonald’s omega. Results EFA revealed a three-factor structure explaining 49.186% of the total variance: Academic Writing Aid (20.438%), Academic Task Support (14.410%), and Reliance and Trust (14.338%). CFA confirmed the three-factor structure with acceptable fit indices (χ2(87) = 223.604, p < .001; CMIN/DF = 2.570; CFI = 0.917; TLI = 0.900; RMSEA = 0.060). All standardized factor loadings were statistically significant (p < .001), ranging from 0.434 to 0.728. The scale demonstrated good internal consistency (Cronbach’s α = 0.848, McDonald’s ω = 0.849) and composite reliability (CR = 0.855). The average variance extracted (AVE) was 0.664, supporting convergent validity. Conclusions The validated ChatGPT Usage Scale provides a reliable and valid instrument for assessing postgraduate students’ engagement with ChatGPT across multiple dimensions. This tool offers valuable insights into AI-assisted academic practices, enabling more nuanced investigations into the effects of ChatGPT on postgraduate education.

Keywords