How stakeholder engagement has led us to reconsider definitions of rigour in systematic reviews

Abstract

Abstract As a methodology designed to inform policy and practice decisions, it is particularly important to ensure that systematic reviews are shaped by those who will use them. There is a broad range of approaches for engagement of the potential users of reviews that aim to elicit their priorities and needs and incorporate these into the review design. This incorporation of their priorities and needs can create a tension between their calls for locally-specific, often rapidly-produced evidence syntheses for policy needs and the production of unbiased, generalisable, globally-relevant systematic reviews. This tension raises the question of what is a ‘gold standard’ review. This commentary aims to address head on this often undiscussed key challenge with regard to stakeholder involvement in systematic reviews: that responding to stakeholders can mean reconsidering what makes a review rigorous. The commentary proposes a new model to address these tensions that combines the production of public-good reviews, with stakeholder-driven syntheses. In this, it presents the approach taken by our team at the Africa Centre for Evidence at the University of Johannesburg to achieve two different but complementary outputs: (i) ‘public goods’, namely comprehensive and generalisable systematic reviews of the evidence available for and accessible to a global audience, and (ii) locally-focussed, stakeholder-driven, pragmatically-produced syntheses for decision-making at a policy level. The designed approach incorporates balancing the formal requirements of full, published systematic reviews with engagement of national and international decision-makers. It also accommodates space to move from stakeholder engagement to co-production, where stakeholders are engaged to such an extent that they become partners in the production of the review. These approaches are integrated into the traditional steps for producing a systematic review with implications as to what constitutes a gold standard approach to synthesising evidence.

Keywords

• Stakeholder engagement
• Rigour
• Rapid evidence assessment
• Public good
• Government