Improving Text Summarization Quality by Combining T5-Based Models and Convolutional Seq2Seq Models

Abstract

In the natural language processing field, there are several sub-fields that are very closely related to information retrieval, such as the automatic text summarization sub-field. obtained from the convolutional T5 and Seq2Seq models in summarizing text on hugging faces found features that can affect text summary such as upper- and lower-case letters which have an impact on changing the understanding of the text of the document. This study uses a combination of parameters such as layer dimensions, learning rate, batch size, and the use of Dropout to avoid model overfitting. The results can be seen by evaluating metrics using ROUGE. This study produces a value of ROUGE-1 on 4 documents that are tested which produces an average of 0.8 which is the optimal value, for ROUGE-2 on 4 documents that are tested which results in an average of 0.83 which is an optimal value while ROUGE-L on 4 documents conducted tests that produce an average of 0.8 which is the optimal value for the summary model.

Keywords

• Model T5
• Seq2Seq Convolutional
• ROUGE