Improving the fairness of ECL listening tests by detecting gender-biased items

Abstract

The first main objective of this study is to examine whether ECL listening tests in German at CEFR level B2 are equally fair to both male and female test takers. The second aim is to explore distinctive patterns that can be associated with gender-related differential item functioning (DIF) and potentially result in gender-biased items. For this purpose, two complementary approaches are used: (1) a multi-faceted Rasch analysis in order to detect items showing DIF in terms of gender, and (2) item characteristics and content analyses in order to identify potential systematic sources of gender-biased items. According to the results of the statistical analysis, only 6.5 per cent of the items show differential item functioning, thus verifying that the ECL German listening tests are generally fair toward the two gender groups. The findings of the study did not show any systematic patterns that can be clearly associated with gender-related DIF or can potentially result in gender-biased items.

Keywords

• test fairness, validity, dif, gender-biased item, ecl german listening tests