Exploring the potential of large language models for integration into an academic statistical consulting service-the EXPOLS study protocol.

Abstract

BackgroundThe advancement of Artificial Intelligence, particularly Large Language Models (LLMs), is rapidly progressing. LLMs, such as OpenAI's GPT, are becoming vital in scientific and medical processes, including text production, knowledge synthesis, translation, patient communication and data analysis. However, the outcome quality needs to be evaluated to assess the full potential for usage in statistical applications. LLMs show potential for all research areas, including teaching. Integrating LLMs in research, education and medical care poses opportunities and challenges, depending on user competence, experience and attitudes.ObjectiveThis project aims at exploring the use of LLMs in supporting statistical consulting by evaluating the utility, efficiency and satisfaction related to the use of LLMs in statistical consulting from both advisee and consultant perspective. Within this project, we will develop, execute and evaluate a training module for the use of LLMs in statistical consulting. In this context, we aim to identify the strengths, limitations and areas for potential improvement. Furthermore, we will explore experiences, attitudes, fears and current practices regarding the use of LLMs of the staff at the Medical Center and the University of Freiburg.Materials and methodsThis multimodal study includes four study parts using qualitative and quantitative methods to gather data. Study part (I) is designed as mixed mode study to explore the use of LLMs in supporting statistical consulting and to evaluate the utility, efficiency and satisfaction related to the use of LLMs. Study part (II) uses a standardized online questionnaire to evaluate the training module. Study part (III) evaluates the consulting sessions using LLMs from advisee perspective. Study part (IV) explores experiences, attitudes, fears and current practices regarding the use of LLMs of the staff at the Medical Center and the University of Freiburg. This study is registered at the Freiburg Registry of Clinical Studies under the ID: FRKS004971.