Construction and Application of Semantic Retrieval Model for Ancient Agricultural Literature

Abstract

[Purpose/Significance] The ancient Chinese agricultural books are the main carrier of traditional agricultural experience, and represent the productivity and the essence of agricultural history in China. The value of agricultural knowledge in them has not disappeared with the progress of the times, and still has practical guidance for the problems that arise in modern agriculture. However, the ancient Chinese agricultural books are written in ancient Chinese, which are obscure and without punctuation, making them difficult to use. Semantic retrieval is a retrieval method that automatically queries and extracts relevant information from information sources at the semantic level. It can accurately capture the true intention behind user problems and conduct searches based on it, and thereby it is capable of returning more accurate and the most consistent results to users. However, currently most relevant research only focuses on major languages, and there is insufficient research on sentence embedding in ancient Chinese prose. In order to fill the gap in the field and provide scholars with more convenient methods for retrieving ancient agricultural knowledge and tracing ancient agricultural knowledge, this study is based on comparative learning methods to construct a semantic retrieval model that can automatically return the most relevant ancient agricultural paragraph with input, using vernacular Chinese as the query. [Method/Process] SikuBERT, which is based on Siku Quanshu as the training corpus, is used as the basic model. Based on the method of comparative learning, the model is continued to be trained using the self-built ancient agricultural dataset, and a semantic retrieval model that can support the use of vernacular as a query and return the ancient agricultural paragraphs most similar to the query semantics is obtained. [Results/Conclusions] The Spearman coefficient of the ancient agricultural text semantic retrieval model can achieve 86.51% performance on the test set, which is a certain degree of improvement compared to the baseline model's 83.69% performance on the test set. The recall situation on the self built ancient agricultural literature retrieval test set has been improved to a certain extent compared to the baseline model, and the model can have good retrieval results on ancient agricultural literature. However, semantic retrieval models usually require relevant semantic similarity datasets or semantic matching datasets for training. Due to the lack of large-scale and pure ancient Chinese data in the field of ancient agricultural literature, and the high cost of constructing relevant datasets requiring personnel with high-standard relevant professional qualifications, this experiment used a self-built dataset for training, which is limited by the quantity and quality of ancient agricultural language corpus data. The current semantic retrieval model for ancient agricultural literature is still not as effective as expected. In the future, we will search for training methods suitable for small samples, such as transfer learning based on cross language pre-training models to improve the retrieval performance.

Keywords

• ancient agricultural script|semantic retrieval|comparative learning|model building|deep learning