Factuality Guided Diffusion-Based Abstractive Summarization

Abstract

Abstractive summarization models are required to generate summaries that maintain factual consistency with the source text and exhibit high diversity to be applicable in practical applications. Existing models, which are based on pre-trained sequence-to-sequence or text diffusion approaches, generally struggle to balance these aspects, as emphasizing one typically compromises the other. To achieve both factual consistency and high diversity in summarization, this paper proposes a factuality-guided diffusion-based abstractive summarization model. This model integrates a factuality-guided module into the diffusion-based model. As the diffusion-based summarization model generates a high-diversity summary by denoising from random noise, the module guides the noise toward factual consistency with the source text. The proposed method continually guides factuality into the intermediate noise at each denoising step, thereby generating summaries that are not only consistent with the source text but also high in diversity. To guide factuality during the denoising step, this study also introduces a method for calculating the factuality based on token-level contextual matching between the source text and the intermediate noise. The effectiveness of the proposed factuality-guided summarization model is validated on three benchmark datasets, and experimental results demonstrate that the summaries generated by the proposed model are more factually consistent and diverse than those generated by baseline models.

Keywords

• Diffusion-based abstractive summarization
• diverse text summarization
• factuality-guided summarization
• factually consistency in abstractive summarization