Application of the Contemporary Psychometrics for Assessing Economic Literacy

Abstract

Currently, new skills and various types of "new literacies" relevant to the modern world are becoming issues of growing importance. One of them is economic literacy; however, there are only few assessment instruments that fulfil the academic requirements for its assessment among university students. One of such internationally established instruments is the Test of Understanding in College Economics (TUCE), which is a popular tool in empirical studies of economic literacy in many countries around the world. Despite its advantages, the currently available version of the TUCE designed for American colleges back in 2006, is prone to cheating and provides limited opportunities for formative feedback. The purpose of this paper is to present the Updated Test of Understanding in College Economics (U-TUCE). In developing the U-TUCE, we utilized the capabilities of contemporary psychometrics, which offer sufficient advances in overcoming all limitations of the original TUCE mentioned before. First, we present a revised theoretical framework of the U-TUCE, highlighting that the test measures different types of mastery of economic literacy. Second, we describe the approaches used for modifying the TUCE items and developing new items. A half of the original test items have been replaced or redesigned to reflect the economic context that has changed since 2006. Third, we utilize the logic of automatic item generation algorithms to increaseg the level of test protection against cheating. We made all changes in such a way as to maintain comparability with the previous versions of the TUCE test if necessary. Finally, the use of the Item Response Theory (IRT) is paired up with that of Cognitive Diagnostic Modeling (CDM) to ensure the quality of the U-TUCE and enhance its formative value. We show that IRT can be used to estimate the construct as a whole (which is of interest to researchers, administrators, and policy makers), while CDM provides information relating to each of the construct components, which are of interest to educational practitioners and students themselves. The results of the data analyses show that the test can be used for both purposes simultaneously.

Keywords

• economic literacy
• psychometric modeling
• item response theory
• cognitive diagnostic modeling
• automated item generation
• formative feedback