Outcomes assessment pitfalls: challenges to quantifying knowledge gain in a sex education game [version 3; peer review: 2 approved, 1 not approved]

Abstract

Background: The use of videogames as a public health tool is rapidly expanding. Accurate assessment of the efficacy of such games is complicated by many factors. We describe challenges associated with measuring the impact of playing a videogame with information about human sexual anatomy and reproduction and discuss motivations for, and solutions to, these challenges. Methods: The My Future Family Game (MFF) is a validated tool for collecting data about family planning intentions which includes information about human anatomy and sexual reproduction. We sought to assess the efficacy of the game as a tool for teaching sexual education using a pre-post model which was deployed in three schools in and around Chennai, India in summer of 2018. Results: The MFF game was successfully modified to collect data about players’ pre-gameplay knowledge of sexual anatomy and processes. The post gameplay assessment process we used did not effectively assess knowledge gain. Designing assessments for games dealing with sexuality presents challenges including: effectively communicating about biological parts and processes, designing usable and intuitive interfaces with minimal text, ensuring that all parts of the process are fun, and integrating assessments into the game in a way that makes them invisible. Conclusion: Games can be an effective means of gathering data about knowledge of sex and reproduction that it is difficult to obtain through other means. Assessing knowledge about human sexual reproduction is complicated by cultural norms and taboos, and technical hurdles which can be addressed through careful design. This study adds to the sparse literature in the field by providing information about pitfalls to avoid and best practices in this evolving area.