Survey on Pre-trained Models Fusing Knowledge Graphs

Abstract

Purpose In practical applications, the pre-trained model still faces the challenge of low quality and quantity of knowledge information required for complex tasks, while the fusion of knowledge graph into the pre-trained model can enhance its performance. Methods In this paper, literatures about knowledge graph fusion pre-training model in recent years have been analyzed and summarized. First, the reasons, advantages, and difficulties of introducing knowledge graph into pre-training model have been introduced briefly. Second, two kinds of methods of implicit combination and explicit combination are discussed in detail, and the characteristics, advantages, and disadvantages of representative models are compared and summarized. Finally, the challenges and future research trends of pre-training models with fusion knowledge graph are discussed. Conclusions The core issue of pre-training models incorporating knowledge graphs is to solve how to effectively integrate information from knowledge bases into the training model. In the future, more effective and efficient knowledge fusion methods can be explored to improve model performance and generalization ability.

Keywords

• deep learning
• pre-training model
• knowledge graph
• enhance