'Stability' of Assessment: Extending the Utility Equation

Abstract

The COVID-19 pandemic has brought many challenges to medical education, necessitating a rapid transition to digital delivery. The widespread move to online exams has introduced novel risks, including the risk of catastrophic IT failure. These are not ‘black swan’ events – something so unexpected and devastating that we could not anticipate them and prepare accordingly. The phrase ‘black elephants’, a cross between a black swan and ‘the elephant in the room’ has been coined to describe these events.   Moving to high-stakes online examinations introduces another element that needs to be considered and managed: the ‘stability’ of the assessment format used. This dimension incorporates notions of ‘platform reliability’ and ‘internal risk management’ and can be caused by both unplanned events eg IT failures, and planned events like security breaches   Developing approaches to mitigate this new risk suggests another dimension to the well-known assessment ‘utility equation’: stability of the platform used. This paper explores the concept of stability from the perspectives of educational institutions and candidates and offers some approaches to achieving stability. The delivery of assessment in the digital age, requires the ‘utility equation’ to be recalibrated and establishment of a new “sweet spot” for each assessment program.  

Keywords

• assessment
• utility equation
• stability