European Urology Open Science (Feb 2024)
Artificial Intelligence: Ready To Pass the European Board Examinations in Urology?
Abstract
The role of artificial intelligence (AI) in the medical domain is increasing on an annual basis. AI allows instant access to the latest scientific data in urological surgery, facilitating a level of theoretical knowledge that previously required several years of practice and training. To evaluate the capability of AI to provide robust data in a specialized domain, we submitted the in-service assessment of the European Board of Urology to three different AI tools: ChatGPT 3.5, ChatGPT 4.0, and Bard. The assessment consists of 100 single-answer questions with four multiple-choice options. We compared the responses of 736 participants to the AI responses. The average score for the 736 participants was 67.20. ChatGPT 3.5 scored 59 points, ranking in 570th place. ChatGPT 4.0 scored 80 points, ranking 80th, just on the border of the top 10%. Google Bard scored 68 points, ranking 340th. Our study demonstrates that AI systems have the capability to participate in a urological examination and achieve satisfactory results. However, a critical perspective must be maintained, as current AI systems are not infallible. Finally, the role of AI in the acquisition of knowledge and the dissemination of information remains to be delineated. Patient summary: We submitted questions from the European Diploma in Urological Surgery to three artificial intelligence (AI) systems. Our findings reveal that AI tools show remarkable performance in assessments of urological surgical knowledge. However, certain limitations were also observed.
