Development and application of the RE-AIM QuEST mixed methods framework for program evaluation

Abstract

To increase the likelihood of successful implementation of interventions and promote dissemination across real-world settings, it is essential to evaluate outcomes related to dimensions other than Effectiveness alone. Glasgow and colleagues' RE-AIM framework specifies four additional types of outcomes that are important to decision-makers: Reach, Adoption, Implementation (including cost), and Maintenance. To further strengthen RE-AIM, we propose integrating qualitative assessments in an expanded framework: RE-AIM Qualitative Evaluation for Systematic Translation (RE-AIM QuEST), a mixed methods framework. RE-AIM QuEST guides formative evaluation to identify real-time implementation barriers and explain how implementation context may influence translation to additional settings. RE-AIM QuEST was used to evaluate a pharmacist-led hypertension management intervention at 3 VA facilities in 2008–2009. We systematically reviewed each of the five RE-AIM dimensions and created open-ended companion questions to quantitative measures and identified qualitative and quantitative data sources, measures, and analyses. To illustrate use of the RE-AIM QuEST framework, we provide examples of real-time, coordinated use of quantitative process measures and qualitative methods to identify site-specific issues, and retrospective use of these data sources and analyses to understand variation across sites and explain outcomes. For example, in the Reach dimension, we conducted real-time measurement of enrollment across sites and used qualitative data to better understand and address barriers at a low-enrollment site. The RE-AIM QuEST framework may be a useful tool for improving interventions in real-time, for understanding retrospectively why an intervention did or did not work, and for enhancing its sustainability and translation to other settings.

Keywords

• RE-AIM
• Mixed methods
• Program evaluation
• Clinical pharmacist intervention