A Delphi study to assess the effect of changes in language between the first and second editions of the WHO’s Joint External Evaluation

Abstract

Introduction Countries use the WHO Joint External Evaluation (JEE) tool—part of the WHO International Health Regulations (2005) Monitoring and Evaluation Framework—for voluntary evaluation of global health security (GHS) capacities. After releasing the JEE first edition (E1) in 2016, WHO released the JEE second edition (E2) in 2018 with language changes to multiple indicators and associated capacity levels. To understand the effect of language changes on countries’ ability to meet requirements in each edition, we conducted a Delphi study—a method where a panel of experts reach consensus on a topic through iterative, anonymous surveys—to solicit feedback from 40+ GHS experts with expertise in one or more of the 19 JEE technical areas.Methods We asked experts first to compare the language changes for each capacity level within each indicator and identify how these changes affected the indicator overall; then to assess the ability of a country to achieve the same capacity level using E2 as compared with E1 using a Likert-style score (1–5), where ‘1’ was ‘significantly easier’ and ‘5’ was ‘significantly harder’; and last to provide a qualitative justification for score selections. We analysed the medians and IQR of responses to determine where experts reached consensus.Results Results demonstrate that 14 indicators and 49 capacity levels would be harder to achieve in E2.Conclusion Findings underscore the importance of considering how language alterations impact how the JEE measures GHS capacity and the feasibility of using the JEE to monitor changes in capacity over time.