Developing a methodological framework for organisational case studies: a rapid review and consensus development process

Abstract

Background: Organisational case study proposals can be poorly articulated and methodologically weak, raising the possible need for publication standards in this area. Objectives: To develop reporting standards for organisational case study research, with particular application to the UK National Health Service. Design: Rapid evidence synthesis and Delphi consensus process. Data sources: Relevant case studies and methods texts were identified through searches of library catalogues, key text and author searches, focused searching of health and social science databases and some targeted website searching. Review methods: The reporting standards were developed in three stages: (1) a rapid review of the existing literature to identify items; (2) a modified Delphi consensus process to develop and refine content and structure; and (3) application of the high-consensus Delphi items to two samples of organisational case studies to assess their feasibility as reporting standards. Items for the Delphi consultation were identified from published organisational case studies and related methodological texts. Identified items were sent to a Delphi expert panel for rating over two rounds. Participants were also asked whether or not the provisional framework in which items were presented was appropriate, and were given the opportunity to adapt this alongside the content. In both rounds, the high-consensus threshold was set at 70% agreement among respondents for each item. High-consensus items from the Delphi consultation were then applied to previously identified case study publications to determine their relevance to the reporting of real-world organisational case studies and to better understand how the results of the Delphi consultation might best be implemented as a reporting standard. Results: One hundred and three unique reporting items were identified from 25 methodological texts; eight example case studies and 12 exemplar case studies did not provide any additional unique items. Thirteen items were ultimately rated as ‘Should be reported for all organisational case studies’ by at least 70% of respondents, with the degree of consensus ranging from 73% to 100%. As a whole, exemplar case studies [which had been provided by the National Institute for Health Research (NIHR)’s Health Services and Delivery Research (HSDR) programme as examples of methodologically strong projects] more consistently reported the high-consensus Delphi items than did case studies drawn from the literature more broadly. Limitations: Time and resource constraints prevented an initial ‘item-generation’ round in the Delphi consensus process. Items are therefore likely to have been influenced by the content, wording and assumptions of available literature. Conclusions: The high-consensus items were translated into a set of 13 reporting standards that aim to improve the consistency, rigour and reporting of organisational case study research, thereby making it more accessible and useful to different audiences. The reporting standards themselves are intended primarily as a tool for authors of organisational case studies. They briefly outline broad requirements for rigorous and consistent reporting without constraining methodological freedom. Future work: These reporting standards should be included as part of the submission requirements for all organisational case studies seeking funding. Though these reporting standards do not mandate specific methods, if a reporting item is not reported for legitimate methodological reasons, the onus is on the author to outline their rationale for the reader. Funding: The NIHR HSDR programme.

Keywords

• reporting standards
• reporting guidelines
• delphi consensus process
• organisational case studies
• research methods