Feasibility of Virtual Focus Groups in Program Impact Evaluation

Abstract

Focus groups are often used for qualitative investigations. We adapted a published focus group method for evaluating impact of an organizational intervention for virtual delivery using video conferencing. The method entailed convening small groups of three to five participants for a 2-hour facilitated workshop. We delivered the virtual workshops, adding qualitative evaluation with researchers and participants, to assess the effectiveness of the protocol. We address the questions of how to structure the data collection procedures; whether virtual delivery permits cross participant interactions about a studied intervention; and how easy and comfortable the experience was for participants. Participants were university faculty members who were the focus of an institutional diversity program. The results indicated that the virtually delivered focus group workshop could be successfully implemented with strong fidelity to the original protocol to achieve the workshop goals. The workshops generated rich data about the impacts of the institutional program as well as other events and conditions in the working environment that were relevant to consider along with the observed program outcomes. A well-planned virtual focus group protocol is a valuable tool to engage intervention stakeholders for research and evaluation from a distance. Video conferencing is especially useful during the current COVID-19 pandemic, but also whenever geography separates researchers and evaluators from program stakeholders. Careful planning of privacy measures for a secure online environment and procedures for structured facilitation of group dialogue are critical for success, as in any focus group. This article addresses a gap in the literature on feasibility and methodology for using video conference technology to conduct qualitative data collection with groups.