The EXpert PANel Decision (EXPAND) method: a way to measure the impact of diverse quality improvement activities of clinical networks

Abstract

Objectives and importance of study: Evaluating impacts of quality improvement activities across diverse clinical focus areas is challenging. However, evaluation is necessary to determine if the activities had an impact on quality of care and resulted in system-wide change. Clinical networks of health providers aim to provide a platform for accelerating quality improvement activities and adopting evidence based practices. However, most networks do not collect primary data that would enable evaluation of impact. We adapted an established expert panel approach to measure the impacts of efforts in 19 clinical networks to improve care and promote health system change, to determine whether these efforts achieved their purpose. Study type: A retrospective cross-sectional study of 19 clinical networks using multiple methods of data collection including the EXpert PANel Decision (EXPAND) method. Methods: Network impacts were identified through interviews with network managers (n = 19) and co-chairs (n = 32), and document review. The EXPAND method brought together five independent experts who provided initial individual ratings of overall network impact. After attendance at an in-person moderated meeting where aggregate scores were discussed, the experts provided a final rating. Median scores of postmeeting ratings were the final measures of network impact. Results: Among the 19 clinical networks, experts rated 47% (n = 9) as having a limited impact on improving quality of care, 37% (n = 7) as having a moderate impact and 16% (n = 3) as having a high impact. The experts rated 26% (n = 5) of clinical networks as having a limited impact on facilitating system-wide change, 37% (n = 7) as having a moderate impact and 37% (n = 7) as having a high impact. Conclusion: The EXPAND method enabled appraisal of diverse clinical networks in the absence of primary data that could directly evaluate network impacts. The EXPAND method can be applied to assess the impact of quality improvement initiatives across diverse clinical areas to inform healthcare planning, delivery and performance. Further research is needed to assess its reliability and validity.