Emergency department triaging using ChatGPT based on emergency severity index principles: a cross-sectional study

Abstract

Abstract Erroneous and delayed triage in an increasingly crowded emergency department (ED). ChatGPT is an artificial intelligence model developed by OpenAI® and is being trained for use in natural language processing tasks. Our study aims to determine the accuracy of patient triage using ChatGPT according to the emergency severity index (ESI) for triage in EDs. In our cross-sectional study, 18 years and over patients who consecutively presented to our ED within 24 h were included. Age, gender, admission method, chief complaint, state of consciousness, and comorbidities were recorded on the case form, and the vital signs were detected at the triage desk. A five-member expert committee (EC) was formed from the fourth-year resident physicians. The investigators converted real-time patient information into a standardized case format. The urgency status of the patients was evaluated simultaneously by EC and ChatGPT according to ESI criteria. The median value of the EC decision was accepted as the gold standard. There was a statistically significant moderate agreement between EC and ChatGPT assessments regarding urgency status (Cohen’s Kappa = 0.659; P < 0.001). The accuracy between these two assessments was calculated as 76.6%. There was a high degree of agreement between EC and ChatGPT for the prediction of ESI-1 and 2, indicating high acuity (Cohen’s Kappa = 0.828). The diagnostic specificity, NPV, and accuracy of ChatGPT were determined as 95.63, 98.17 and 94.90%, respectively, for ESI high acuity categories. Our study shows that ChatGPT can successfully differentiate patients with high urgency. The findings are promising for integrating artificial intelligence-based applications such as ChatGPT into triage processes in EDs.

Keywords

• Artificial intelligence
• Emergency department
• Emergency severity index
• Triage