To be fair: Development and illustration of the Comprehensive Appraisal of Fairness Evidence (CAFE) model to advance SEL assessment practices

Abstract

The use of school-based universal assessment of students’ social and emotional competencies is increasing with the popularity of SEL programs (Collaborative for the Academic, Social, and Emotional Learning, 2020). Likewise, the development of social emotional assessments has increased in technical sophistication. Yet, a fundamental aspect of their use and interpretation – fairness – has remained relatively unexamined. One potential reason for this situation is the lack of an integrative framework that provides guidance regarding the types of evidence necessary for fair assessment scores. To address this need for fairness evidence, we propose a new model - the Comprehensive Appraisal of Fairness Evidence (CAFE). This model features a unified evidence framework with three facets based upon a targeted expansion and integration of Kane’s (2010) notion of procedural and substantive fairness and the Standards for Educational and Psychological Testing (AERA, APA, & NCME, 2014). This framework is embedded within an evaluation process that grounds consideration of fairness in specific proposed interpretations and uses. We then provide an example application of this model to illustrate it utility with a published measure of social emotional competence, the SSIS SEL Brief Scales (Elliott et al., 2020). We conclude with implications for social emotional assessment developers, educators, and administrators working to ensure fair assessment of all students. Impact Statement: Fairness is a human quality taught in many K-12 SEL programs. Fairness also is an expected quality of tests and assessments. Ironically, few authors of assessments report evidence to support the fairness of their scores for specific uses. Thus, we developed the Comprehensive Appraisal of Fairness Evidence (CAFE) Model to advance consideration of the fairness of SEL assessment scores for all students. This model consists of three types of evidence: Procedural, Contextual, and Consequential. Examples of evidence illustrate the use of this new integrative model that supports a key goal of child SEL assessment development and use – to be fair!

Keywords

• CAFE model
• Fairness
• Test use and interpretation
• Test bias
• SEL assessment