ChatGPT-3.5 System Usability Scale early assessment among Healthcare Workers: Horizons of adoption in medical practice
Fadi Aljamaan,
Khalid H. Malki,
Khalid Alhasan,
Amr Jamal,
Ibraheem Altamimi,
Afnan Khayat,
Ali Alhaboob,
Naif Abdulmajeed,
Fatimah S. Alshahrani,
Khaled Saad,
Ayman Al-Eyadhy,
Jaffar A. Al-Tawfiq,
Mohamad-Hani Temsah
Affiliations
Fadi Aljamaan
College of Medicine, King Saud University, Riyadh 11362, Saudi Arabia; Critical Care Department, College of Medicine, King Saud University, Riyadh 11362, Saudi Arabia
Khalid H. Malki
Research Chair of Voice, Swallowing, and Communication Disorders, Department of Otolaryngology, College of Medicine, King Saud University, Riyadh 11362, Saudi Arabia
Khalid Alhasan
College of Medicine, King Saud University, Riyadh 11362, Saudi Arabia; Pediatric Department, College of Medicine, King Saud University Medical City, Riyadh 11362, Saudi Arabia; Department of Kidney and Pancreas Transplant, Organ Transplant Center of Excellence, King Faisal Specialist Hospital and Research Center, Riyadh 11211, Saudi Arabia
Amr Jamal
College of Medicine, King Saud University, Riyadh 11362, Saudi Arabia; Department of Family and Community Medicine, King Saud University Medical City, Riyadh 11362, Saudi Arabia; Evidence-Based Health Care & Knowledge Translation Research Chair, Family & Community Medicine Department, College of Medicine, King Saud University, Riyadh 11362, Saudi Arabia
Ibraheem Altamimi
College of Medicine, King Saud University, Riyadh 11362, Saudi Arabia
Afnan Khayat
Health Information Management Department, Prince Sultan Military College of Health Sciences, Al Dhahran 34313, Saudi Arabia
Ali Alhaboob
College of Medicine, King Saud University, Riyadh 11362, Saudi Arabia; Pediatric Department, College of Medicine, King Saud University Medical City, Riyadh 11362, Saudi Arabia
Naif Abdulmajeed
Pediatric Department, College of Medicine, King Saud University Medical City, Riyadh 11362, Saudi Arabia; Pediatric Nephrology Department, Prince Sultan Military Medical City, Riyadh 11159, Saudi Arabia
Fatimah S. Alshahrani
College of Medicine, King Saud University, Riyadh 11362, Saudi Arabia; Infectious Disease Division, Department of Medicine, King Saud University Medical City, Riyadh 11362, Saudi Arabia
Khaled Saad
Pediatric Department, Faculty of Medicine, Assiut University, Assiut 71516, Egypt
Ayman Al-Eyadhy
College of Medicine, King Saud University, Riyadh 11362, Saudi Arabia; Pediatric Department, College of Medicine, King Saud University Medical City, Riyadh 11362, Saudi Arabia
Jaffar A. Al-Tawfiq
Specialty Internal Medicine and Quality Department, Johns Hopkins Aramco Healthcare, Dhahran 34465, Saudi Arabia; Infectious Disease Division, Department of Medicine, Indiana University School of Medicine, Indianapolis, IN46202, USA; Infectious Disease Division, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, MD21218, USA
Mohamad-Hani Temsah
College of Medicine, King Saud University, Riyadh 11362, Saudi Arabia; Pediatric Department, College of Medicine, King Saud University Medical City, Riyadh 11362, Saudi Arabia; Evidence-Based Health Care & Knowledge Translation Research Chair, Family & Community Medicine Department, College of Medicine, King Saud University, Riyadh 11362, Saudi Arabia; Corresponding author. College of Medicine, King Saud University, Riyadh, 11362, Saudi Arabia.
Artificial intelligence (AI) chatbots, such as ChatGPT, have widely invaded all domains of human life. They have the potential to transform healthcare future. However, their effective implementation hinges on healthcare workers’ (HCWs) adoption and perceptions. This study aimed to evaluate HCWs usability of ChatGPT three months post-launch in Saudi Arabia using the System Usability Scale (SUS).A total of 194 HCWs participated in the survey. Forty-seven percent were satisfied with their usage, 57 % expressed moderate to high trust in its ability to generate medical decisions. 58 % expected ChatGPT would improve patients’ outcomes, even though 84 % were optimistic of its potential to improve the future of healthcare practice. They expressed possible concerns like recommending harmful medical decisions and medicolegal implications.The overall mean SUS score was 64.52, equivalent to 50 % percentile rank, indicating high marginal acceptability of the system. The strongest positive predictors of high SUS scores were participants' belief in AI chatbot's benefits in medical research, self-rated familiarity with ChatGPT and self-rated computer skills proficiency. Participants' learnability and ease of use score correlated positively but weakly. On the other hand, medical students and interns had significantly high learnability scores compared to others, while ease of use scores correlated very strongly with participants' perception of positive impact of ChatGPT on the future of healthcare practice.Our findings highlight the HCWs' perceived marginal acceptance of ChatGPT at the current stage and their optimism of its potential in supporting them in future practice, especially in the research domain, in addition to humble ambition of its potential to improve patients’ outcomes particularly in regard of medical decisions. On the other end, it underscores the need for ongoing efforts to build trust and address ethical and legal concerns of AI implications in healthcare. The study contributes to the growing body of literature on AI chatbots in healthcare, especially addressing its future improvement strategies and provides insights for policymakers and healthcare providers about the potential benefits and challenges of implementing them in their practice.
