Diagnosis Unreliability of ChatGPT for Journal Evaluation

Abstract

Purpose: Academic and other researchers have limited tools with which to address the current proliferation of predatory and hijacked journals. These journals can have negative effects on science, research funding, and the dissemination of information. As most predatory and hijacked journals are not error free, this study used ChatGPT, an artificial intelligence (AI) technology tool, to conduct an evaluation of journal quality. Methods: Predatory and hijacked journals were analyzed for reliability using ChatGPT, and the reliability of result have been discussed. Results: It shows that ChatGPT is an unreliable tool for journal quality evaluation for both hijacked and predatory journals. Conclusion: To show how to address this gap, an early trial version of Journal Checker Chatbot has been developed and is discussed as an alternative chatbot that can assist researchers in detecting hijacked journals.

Keywords

• hijacked journals
• predatory journals
• chatgpt
• artificial intelligence
• language models
• research ethics