A Novel Approach for Mixed‐Methods Research Using Large Language Models: A Report Using Patients’ Perspectives on Barriers to Arthroplasty

Abstract

Objective Mixed‐methods research is valuable in health care to gain insights into patient perceptions. However, analyzing textual data from interviews can be time‐consuming and require multiple analysts for investigator triangulation. This study aims to explore a novel approach to investigator triangulation in mixed‐methods research by employing a large language model (LLM) for analyzing data from patient interviews. Methods This study compared the thematic analysis and survey generation performed by human investigators and ChatGPT‐4, which uses GPT‐4 as its backbone model, using data from an existing study that explored patient perceptions of barriers to arthroplasty. The human‐ and ChatGPT‐4–generated themes and surveys were compared and evaluated based on their representation of salient themes from a predetermined topic guide. Results ChatGPT‐4 generated analogous dominant themes and a comprehensive corresponding survey as the human investigators but in significantly less time. The survey questions generated by ChatGPT‐4 were less precise than those developed by human investigators. The mixed‐methods flowchart proposes integrating LLMs and human investigators as a supplementary tool for the preliminary thematic analysis of qualitative data and survey generation. Conclusion By utilizing a combination of LLMs and human investigators through investigator triangulation, researchers may be able to conduct more efficient mixed‐methods research to better understand patient perspectives. Ethical and qualitative implications of using LLMs should be considered.