The versatility of generalizability theory as a tool for exploring and controlling measurement error

Abstract

Measurement error arises from many sources in educational assessment. It is important to estimate the importance of this error, and, if appropriate, to seek ways to reduce it. Generalizability theory represents a powerful tool in this sense, allowing identifiable error contributions to be separately quantified, and measurement error to be estimated and even predicted in response to possible changes in the measurement procedure. The paper offers examples of generalizability analysis of numeracy attainment data deriving from the Scottish Survey of Achievement, with the aim of illustrating the versatility of the methodology for error estimation and prediction in this type of sample-based programme.

Keywords

• Generalizability theory
• national assessment
• attainment surveys
• numeracy assessment
• test reliability
• domain sampling
• matrix sampling
• Théorie de la généralisabilité
• évaluation nationale
• enquêtes pour le suivi des acquis
• évaluation des notions de calcul
• fiabilité d’un test
• échantillonnage par domaine
• échantillonnage matriciel
• Teoria da generalizabilidade
• avaliação nacional
• estudos das aprendizagens adquiridas
• avaliação das noções de cálculo
• fiabilidade de um teste
• amostragem por domínio
• amostragem matricial