Meta AI literacy scale: Further validation and development of a short version

Abstract

The concept of AI literacy, its promotion, and measurement are important topics as they prepare society for the steadily advancing spread of AI technology. The first purpose of the current study is to advance the measurement of AI literacy by collecting evidence regarding the validity of the Meta AI Literacy Scale (MAILS) by Carolus and colleagues published in 2023: a self-assessment instrument for AI literacy and additional psychological competencies conducive for the use of AI. For this purpose, we first formulated the intended measurement purposes of the MAILS. In a second step, we derived empirically testable axioms and subaxioms from the purposes. We tested them in several already published and newly collected data sets. The results are presented in the form of three different empirical studies. We found overall evidence for the validity of the MAILS with some unexpected findings that require further research. We discuss the results for each study individually and also together. Also, avenues for future research are discussed. The study's second purpose is to develop a short version (10 items) of the original instrument (34 items). It was possible to find a selection of ten items that represent the factors of the MAILS and show a good model fit when tested with confirmatory factor analysis. Further research will be needed to validate the short scale. This paper advances the knowledge about the validity and provides a short measure for AI literacy. However, more research will be necessary to further our understanding of the relationships between AI literacy and other constructs.