Bulgarian Journal of Science and Education Policy (Dec 2020)
Efficacy of Item Response Theory in the Validation and Score Ranking of Dichotomous Response Mathematics Achievement Test
Abstract
The authors investigated the efficacy of Item Response Theory in the validation and score ranking of dichotomous response Mathematics Achievement Test. The study employed scale development research type of counterbalance design. The sample consisted of 1080 senior secondary schools three from 36 schools, who were drawn randomly from the Osun East senatorial district of Osun State. Two instruments with empirical reliability of 0.94 and 0.81 were used. Data obtained were subjected to Stout's Test of Essential Unidimensionality, Chen-Thissen LD, paired sample t-test and percentile rank. The results revealed that constructed multiple-choice items fulfilled the assumptions of IRT. Paired-samples t-test for difficulty and discrimination indices of the developed MAT and the NECO test items under CTT showed that their mean difference was statistically significant (t = 2.63, df = 59, P = 0.01) and (t = 12.19, df = 59, P = 0.00) while under IRT, the same trend was observed. Also, under CTT and IRT, the ability estimates showed a statistically significant mean difference between the two tests and the IRT percentile score ranking produced distinctive ranking for testees' who have a similar score ranking under CTT. It was concluded that the IRT method was more effective than CTT in test development and scoring. Examining bodies should calibrate their multiple-choice test using IRT.
