Usefulness of the large language model ChatGPT (GPT‐4) as a diagnostic tool and information source in dermatology

Abstract

Abstract Background The field of artificial intelligence is rapidly evolving. As an easily accessible platform with vast user engagement, the Chat Generative Pre‐Trained Transformer (ChatGPT) holds great promise in medicine, with the latest version, GPT‐4, capable of analyzing clinical images. Objectives To evaluate ChatGPT as a diagnostic tool and information source in clinical dermatology. Methods A total of 15 clinical images were selected from the Danish web atlas, Danderm, depicting various common and rare skin conditions. The images were uploaded to ChatGPT version GPT‐4, which was prompted with ‘Please provide a description, a potential diagnosis, and treatment options for the following dermatological condition’. The generated responses were assessed by senior registrars in dermatology and consultant dermatologists in terms of accuracy, relevance, and depth (scale 1–5), and in addition, the image quality was rated (scale 0–10). Demographic and professional information about the respondents was registered. Results A total of 23 physicians participated in the study. The majority of the respondents were consultant dermatologists (83%), and 48% had more than 10 years of training. The overall image quality had a median rating of 10 out of 10 [interquartile range (IQR): 9–10]. The overall median rating of the ChatGPT generated responses was 2 (IQR: 1–4), while overall median ratings in terms of relevance, accuracy, and depth were 2 (IQR: 1–4), 3 (IQR: 2–4) and 2 (IQR: 1–3), respectively. Conclusions Despite the advancements in ChatGPT, including newly added image processing capabilities, the chatbot demonstrated significant limitations in providing reliable and clinically useful responses to illustrative images of various dermatological conditions.

Keywords

• AI
• artificial intelligence
• Chatbot
• ChatGPT
• clinical dermatology
• GPT‐4