Analysis of influence of the item total scores on the levels of ability of respondents in the dichotomous Rasch model based on the weighted maximum likelihood method

Abstract

Purpose of the study. Rasch model is often used in processing test results. However, when using this model and the maximum likelihood method (ML), the estimates the levels of ability of respondents depend only on the number of correctly performed test items and do not depend on the difficulties of the items. The purpose of the research is to analyze the influence of the difficulties of the items on the levels of abilities of the respondents based on the weighted maximum likelihood method (WML). To obtain the weights of the WML, the item total scores are used. Materials and methods. The analysis of the influence of the difficulties of the items on the levels of abilities of the respondents is investigated using the dichotomous table obtained when testing the knowledge of 19 respondents in the course “Fundamentals of Electronics”. Indicator variables of 16 test items were used. For items, we calculate the item total scores that determine their items difficulties. The weighting coefficients of the used WML depend on the item total scores and on the coefficient of influence K. When K = 0, WML convert into ML. As K increases from 0 to 2, the weighting coefficients increase and it becomes possible to analyze in detail the influence of the difficulties of the items on the respondents’ ability levels. To calculate the parameters of the Rasch model based on WML, programs (M-files) for the MATLAB environment and Ministep (Winsteps) are used. Results. The use of WML with weighting coefficient obtained on the basis of the item total scores of the difficulties of the items allowed us to further differentiate the levels of respondents’ abilities in the dichotomous Rasch model. The results of the analysis performed using the data of the test on electronics show that, ceteris paribus, new levels of person’s abilities increase if respondents perform difficult items and, conversely, the respondent’s ability levels decrease if respondents perform light items. At the same time, the difficulty levels of the items practically do not change. As a rule, the greater the coefficient of influence K, the more different the estimation of abilities of respondents, obtained on the basis of WML, from the estimation on the basis of ML. However, there are respondents whose ability level does not change or change slightly when the coefficient K is increased from 0 to 2. For the data of the test on electronics with a coefficient K ≤ 1, the original order of respondents in their ability levels calculated on the basis of ML is preserved. With an increased coefficient of influence K ≥ 1,5, new levels of ability, calculated using WML, cause a change in the order of distribution of respondents according to ability levels. Calculations performed using the MATLAB package are confirmed by data obtained using the Winsteps program. Differences without extreme respondents do not exceed 0.01 logit with the maximum value of the coefficient K equal to 2. Conclusion. On the basis of WML, a method is proposed for taking into account the influence of the difficulties of items on the levels of respondents’ abilities in the Rasch dichotomous model when using the item total scores. The results of the analysis performed using the data of the test on electronics show that in this case we will obtain a differentiation of the levels of abilities of the respondents who score the same points. Note that the results obtained using WML and using the data of the test on electronics do not reject the data obtained on the basis of the classical dichotomous Rasch model and ML. The results obtained on the basis of WML, allow to refine the levels of abilities of the respondents, obtained on the basis of ML.

Keywords

• модель раша
• взвешенный метод максимального правдоподобия
• весовые коэффициенты
• программа winsteps