Practical Applications of a Participatory Science Project Evaluation Tool: Perspectives from Across Earth and Space Science

Abstract

While traditional science often focuses on peer-reviewed scientific publications and citations to gauge efficacy, citizen science projects, also called participatory science projects, span a broader range of goals, methods, and products than are capturable by such means. The Science Products Inventory (SPI) is one tool to assist with participatory science evaluation that extends beyond scientific outputs such as publications. Its use cases were limited to a cohort of projects during its initial development, so here we explored its utility and flexibility for use cases across multiple US federally-funded participatory science projects. We examined the tool’s formative and longitudinal utility, accessibility, and adaptability with contributory projects spanning different sizes, scopes, and goals. We found that the tool was easily customizable, making it suitable for individual project needs, including meeting the needs of funders for evaluation. It was easy to use, making it suitable for data gathering by non-expert evaluators, and it enabled practitioners to reflect on ways to enhance a project’s impact. We also found it useful for comparing evaluation data over time and between projects. Ultimately, citizen science includes elements across various dimensions that can be examined using quantitative indicators—like those in the SPI—and qualitative means. A combination of evaluation approaches could be appropriate for individual project goals and stakeholders, but we show how the simplicity and customizability of the SPI can provide benefits across a range of projects and programs.

Keywords

• evaluation
• metrics
• scientific products
• data practices
• inventory