Knowledge Graph Completion Using a Pre-Trained Language Model Based on Categorical Information and Multi-Layer Residual Attention

Abstract

Knowledge graph completion (KGC) utilizes known knowledge graph triples to infer and predict missing knowledge, making it one of the research hotspots in the field of knowledge graphs. There are still limitations in generating high-quality entity embeddings and fully understanding the contextual information of entities and relationships. To overcome these challenges, this paper introduces a novel pre-trained language model-based method for knowledge graph completion that significantly enhances the quality of entity embeddings by integrating entity categorical information with textual descriptions. Additionally, this method employs an innovative multi-layer residual attention network in combination with PLMs, deepening the understanding of the joint contextual information of entities and relationships. Experimental results on the FB15k-237 and WN18RR datasets demonstrate that our proposed model significantly outperforms existing baseline models in link prediction tasks.

Keywords

• knowledge graph completion
• pre-trained language models (PLMs)
• textual information
• categorical information
• multi-layer residual attention