Dual-Level Contrastive Learning for Improving Conciseness of Summarization

Abstract

The task of text summarization aims to provide highly condensed summaries of long textual information, with the ideal summary being both precise and concise. In recent years, there has been extensive research on the brevity of summaries, but these methods still have significant room for improvement in ROUGE scores, especially when the beam width is increased. We propose a new model called the DC (Dual-Level Contrastive Learning), which combines contrastive learning and data augmentation, and design a new scoring function during the training phase to enhance accuracy and conciseness. Ultimately, our framework achieves excellent ROUGE scores, ensuring concise and readable output even with increased beam width. Experimental results on the CNN/DailyMail (47.82 ROUGE-1, 0.017 VAR) and XSum (47.31 ROUGE-1, 0.0052 VAR) datasets demonstrate that our approach can significantly enhance the accuracy and conciseness of the summaries. Some metrics have exceeded those of the current state-of-the-art model BRIO, promoting the state-of-the-art performance to a higher level.

Keywords

• Contrastive learning
• data augmentation
• conciseness
• KL-divergence
• text summarization