The promise and challenges of ChatGPT in community pharmacy: A comparative analysis of response accuracy

Abstract

This study evaluates ChatGPT, an AI-based language model, in addressing common pharmacist inquiries in community pharmacies. The assessment encompasses Drug-Drug Interactions, Adverse Drug Effects, Drug Dosage, and Alternative Therapies, each comprising 20 questions, totaling 80 questions. Responses from ChatGPT were compared against standard answers, generating textual and chart scores. Textual score was computed by relating correct answers to the total questions within each category, while chart score involved the total correct answers multiplied by the chart-type questions. ChatGPT exhibited distinct performance rates: 30% for Drug-Drug Interactions, 65% for Adverse Drug Effects, 35% for Drug Dosage, and an impressive 85% for Alternative Therapies. While Alternative Therapies displayed high accuracy, challenges arose in accurately addressing Drug Dosage and Drug-Drug Interactions. Conclusion: The study underscores the complexity of pharmacy-related inquiries and the necessity for AI model enhancement. Despite promising accuracy in certain categories, like Alternative Therapies, improvements are crucial for Drug Dosage and Drug-Drug Interactions. The findings emphasize the need for ongoing AI model development to optimize integration into community pharmacy settings.