A Topic Information Fusion and Semantic Relevance for Text Summarization

Abstract

With the continuous development of deep learning, pre-trained models have achieved sound effects in the field of natural language processing. However, text summarization research is far from what people want, especially in abstractive summarization. A high-quality summarization system needs to focus on the topic content of the document and the similarity between the summary and the source document. In this paper, we propose a topic information fusion and semantic relevance for text summarization based on Fine-tuning BERT(TIF-SR). Primarily, considering the critical role of topic information in summary generation, we extract topic keywords and fusion them with source documents as part of the input. Secondly, make the summary closer to the source document by calculating the semantic similarity between the generated summary and the source document, the quality of the abstract is improved. The experimental data indicate that the ROUGE index and readability have improved in this model, so these shreds of evidence suggest that the method proposed by our model is sufficient.

Keywords

• Summarization generation
• fine-tune BERT
• topic information
• semantic relevance
• transformer
• attention mechanism