Item- and person-level factors in test-taking disengagement: Multilevel modelling in a low-stakes context

Abstract

The present study examines item- and person-level factors that influence test-taking disengagement. Computer-based measurement of complex problem-solving was used to eliminate the effect of factual knowledge on test performance among first-year university students in a low-stakes context. Due to the hierarchical structure of the data, multilevel modeling was used to identify item- and person-level factors that influence test-taking disengagement. Results suggested that item position and item difficulty have a significant effect on test-taking disengagement. Items presented later in test administration as well as more difficult items had a higher probability of disengaged responses. Mother's education had no significant effect on the rate of disengaged responses, while a higher proportion of disengaged responses was recorded among women. The percentage of disengaged responses was also greater among those with lower entrance scores, lower working memory capacity and lower self-reported effort (SRE). To sum up, the results suggest a relationship between the level of academic ability and test-taking disengagement, which determines how disengaged responses are treated.

Keywords

• Test-taking effort
• Logfile analysis
• Response time
• Test-taking disengagement
• Multilevel modeling