Use of differential item functioning (DIF) analysis for bias analysis in test construction

Abstract

When differential item functioning (DIF) item analysis procedures based on item response theory (IRT) are used during test construction, it is possible to draw item characteristic curves for the same item for different subgroups. These curves indicate how each item functions at various ability levels for different subgroups. DIF is indicated by the area between the curves. In the construction of the Learning Potential Computerised Adaptive Test (LPCAT), this method was used to identify items that indicated bias in terms of gender, culture, language or level of education. Items that exceeded a predetermined amount of DIF were discarded from the final item bank, irrespective of which subgroup was being advantaged or disadvantaged. The process and results of the DIF analysis are discussed. Opsomming Waar differensiële itemfunksioneringsprosedures (DIF-prosedures) vir itemontleding gebaseer op itemresponsteorie (IRT) tydens toetskonstruksie gebruik word, is dit moontlik om itemkarakteristiekekrommes vir dieselfde item vir verskillende subgroepe voor te stel. Hierdie krommes dui aan hoe elke item vir die verskillende subgroepe op verskillende vermoënsvlakke te funksioneer. DIF word aangetoon deur die area tussen die krommes. DIF is in die konstruksie van die 'Learning Potential Computerised Adaptive test (LPCAT)' gebruik om die items te identifiseer wat sydigheid ten opsigte van geslag, kultuur, taal of opleidingspeil geopenbaar het. Items wat ’n voorafbepaalde vlak van DIF oorskry het, is uit die finale itembank weggelaat, ongeag die subgroep wat bevoordeel of benadeel is. Die proses en resultate van die DIF-ontleding word bespreek.

Keywords

• Differential item functioning (DIF) analysis
• Test construction