Foot & Ankle Surgery: Techniques, Reports & Cases (Jan 2024)
Investigating the proficiency of an AI tool in summarizing foot and ankle literature: A quantitative, qualitative and accuracy analysis
Abstract
Artificial Intelligence (AI) is rapidly transforming various sectors of industry, including the medical community, due to its ability to simulate human intelligence processes. This study evaluates the capacity of an AI tool, ChatGPT 3.5, to summarize scientific papers in the foot and ankle surgery literature, comparing its performance to summaries written by podiatric surgery residents. Quantitative and Qualitative analyses were performed, including BLEU and ROUGE metrics, Flesch Reading Ease Score (FRES), Flesch-Kincaide Grade Level (FKGL) readability statistics, qualitative analysis by independent reviewers, and independent accuracy assessments. Results indicate that AI-generated summaries closely resemble those produced by podiatric surgery residents in terms of content, readability, and accuracy. Although differing slightly from their human-generated counterparts with a higher level of writing, AI-generated summaries offer consistency and conciseness. Overall, this study demonstrates the potential of AI tools to streamline research processes while emphasizing the importance of judicious use and oversight to maintain scholarly integrity.
