Application of ChatGPT in multilingual medical education: How does ChatGPT fare in 2023's Iranian residency entrance examination

Hamid Khorshidi; Afshin Mohammadi; David M. Yousem; Jamileh Abolghasemi; Golnoosh Ansari; Mohammad Mirza-Aghazadeh-Attari; U Rajendra Acharya; Ali Abbasian Ardakani

Informatics in Medicine Unlocked (Jan 2023)

Application of ChatGPT in multilingual medical education: How does ChatGPT fare in 2023's Iranian residency entrance examination

Hamid Khorshidi,
Afshin Mohammadi,
David M. Yousem,
Jamileh Abolghasemi,
Golnoosh Ansari,
Mohammad Mirza-Aghazadeh-Attari,
U Rajendra Acharya,
Ali Abbasian Ardakani

Affiliations

Hamid Khorshidi: Department of Information Engineering, University of Padova, Padova, Italy
Afshin Mohammadi: Department of Radiology, Faculty of Medicine, Urmia University of Medical Science, Urmia, Iran
David M. Yousem: Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins Medical Institutions, Baltimore, MD, USA
Jamileh Abolghasemi: Department of Biostatistics, School of Public Health, Iran University of Medical Sciences, Tehran, Iran
Golnoosh Ansari: Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins Medical Institutions, Baltimore, MD, USA; Department of Radiology, Northwestern University Feinberg School of Medicine, 420 E Superior St, Chicago, IL, USA
Mohammad Mirza-Aghazadeh-Attari: Russell H. Morgan Department of Radiology and Radiological Science, Johns Hopkins Medical Institutions, Baltimore, MD, USA
U Rajendra Acharya: School of Mathematics, Physics and Computing, University of Southern Queensland, Springfield, Queensland, Australia
Ali Abbasian Ardakani: Department of Radiology Technology, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran; Corresponding author. Department of Radiology Technology, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, P.O.Box: 1971653313, Tehran, Iran.

Journal volume & issue: Vol. 41
p. 101314

Abstract

Read online

Background: ChatGPT is a large language model (LLM) artificial intelligence instrument trained on massive amounts of text data extracted from the internet and/or user input. In the present article, we aim to apply the latest version of ChatGPT to the Iranian Medical Residency Examination. Methods: The Iranian Medical Residency Examination is composed of 200 multichoice questions covering all domains of medicine. We used ChatGPT to translate questions into English, French, and Spanish. We fed the questions as multiple-choice questions and allowed ChatGPT to provide comprehensive answers and justifications for its choices. Results: ChatGPT was able to answer 161 (81.3% = 161/198) questions correctly when the Persian language was used. When the questions were translated into English, French, and Spanish, ChatGPT answered six, one, and five additional questions correctly, respectively. When comparing the different languages, there was no significant difference in the functioning of ChatGPT in different languages using either the McNemar test or the Binomial test. Conclusion: ChatGPT can deliver above-average performance in the Iranian Medical Residency Examination, demonstrating its potential for using language models in medicine.

Published in Informatics in Medicine Unlocked

ISSN: 2352-9148 (Online)
Publisher: Elsevier
Country of publisher: United Kingdom
LCC subjects: Medicine: Medicine (General): Computer applications to medicine. Medical informatics
Website: https://www.journals.elsevier.com/informatics-in-medicine-unlocked/

About the journal

Abstract

Keywords