Nongye tushu qingbao xuebao (Jun 2023)
Application of Large-scale Pre-Training Language Model in Network Health Information Identification
Abstract
[Purpose/Significance] Taking the popular "chat robot" ChatGPT and the recently launched similar product "iFLYTEK Spark" as the research object, this paper explores their applications in the identification of online health information, and discusses their advantages and disadvantages, in order to provide reference for the large-scale pre-training language model in the field of health information identification. Based on the review of relevant literature on online health information authentication, deep learning models have been widely applied in the task of online health information authentication in recent years. With the rapid development of large pre-training language models such as ChatGPT, it is a novel idea to explore their discriminating ability in online health information. [Method/Process] Researchers selected health-related information from the most authoritative rumor-refuting websites in China, used "ChatGPT" and "iFLYTEK Spark" to verify the authenticity of the online health information, evaluated their performance, and compared their identification results with the expert identification results. The identification accuracy of ChatGPT and iFLYTEK Spark language model was 93.9% and 92.9%, respectively, and the F1 value was 0.951 and 0.946, respectively, which had a good application effect. The generated explanatory texts were more detailed and the language was relatively smooth. In terms of the length and dispersion of the explanatory text, ChatGPT is closer to that of medical experts, while iFLYTEK Spark's explanatory text is relatively long and less discrete. In terms of semantic similarity, ChatGPT and iFLYTEK Spark were almost equal in performance, and their understanding of health information was close to that of human experts to some extent. Through the analysis of typical samples, it can be seen that an AI large model cannot accurately identify news or emergency information for the time being, and the understanding of individual health propositions with complex semantics will occasionally be biased. [Results/Conclusions] The experimental results show that ChatGPT and iFLYTEK Spark have good discriminative effect in the field of online health information identification, but there are shortcomings, and manual intervention is needed to ensure the accuracy and reliability of the results. In the future, in the field of AI large model research, researchers are suggested to attach importance to the construction and application of high-quality corpora in vertical fields. In the field of online health information identification, practitioners can use models such as ChatGPT as tools to help identify and refine health information. There are also limitations in this article. For example, the amount of data involved in the test is not large enough, ChatGPT uses GPT3.5 model, and the online application time of iFLYTEK Spark language model is relatively short. In future studies, the amount of online health information can be further increased, and the updated version of an AI large model can be tested and evaluated.
Keywords
