Evaluating Computerized Adaptive Testing Efficiency in Measuring Students’ Performance in Science TIMSS

Abstract

The current standards of assessment are demanding for high level of precision with less time-consuming and personalized opportunities, thus restrict the function of Paper and Pencil Test which has dominated the assessment field for a very long time. The Computerized Adaptive Testing (CAT) is viewed as an alternative testing tool to Paper and Pencil Test as it has adaptive feature which enables it to meet the current standards of assessment. This research is focusing on the evaluation of students’ ability in Grade 8 Science Trend in International Mathematics and Science Study (TIMSS) using Computerized Adaptive Testing (CAT) as an alternative instrument to Paper and Pencil Test to investigate whether the implementation of CAT can produce high level of precision with fewer items administered as well as differentiate different academic level among groups of students. CAT was configured in Concerto and was administered on Form 2 and Form 4 students selected through purposive sampling method from secondary schools in northern part of Malaysia. Students’ performance was analysed and compared in terms of score (theta value), SEM value and their response toward the items selected in CAT using SPSS. Finding shows that the administration of 20 objective items in fixed length CAT produced SEM ≤0.50 indicated that the implemented CAT increased the efficiency of assessment with fewer item administration. The t test showed that there was a significance difference between the two groups’ scores in CAT in which Form 4 students had higher ability level than Form 2 students proving that CAT’s configuration had been done correctly in Concerto and the test was more suitable in challenging Form 2 students’ Science knowledge thus the instrument fulfilled the known-group validity.

Keywords

• ability, computerized adaptive testing (cat), evaluation, timss