SumLLaMA: Efficient Contrastive Representations and Fine-Tuned Adapters for Bug Report Summarization

Abstract

In software maintenance, concise summaries of bug reports are crucial, significantly enhancing developer efficiency and ultimately improving software quality and user experience. Large language models (LLMs) have become the standard method for bug report summarization due to their powerful representation capabilities. However, LLM-based approaches face two primary challenges: accurately modeling the contextual relationships between various components within a bug report and the risk of overfitting when fine-tuning LLMs on datasets of limited size. To address these challenges, we propose a novel approach, SumLLaMA, which leverages contrastive learning pre-training and parameter-efficient fine-tuning. Contrastive learning pre-training is employed to construct contextual relations between components in a single bug report, enabling SumLLaMA to learn sequence-level representations. For parameter-efficient fine-tuning, we fine-tune a smaller adapter instead of the entire LLM, reducing the number of parameters trained to about 1/1500 of the original model, effectively mitigating the risk of overfitting. To evaluate the effectiveness of SumLLaMA, we compare it against five baseline models, including a state-of-the-art model, on a publicly available dataset. The experimental results show that SumLLaMA outperforms all baselines by up to 26.66, 17.10, and 24.01 points in ROUGE-1, ROUGE-2, and ROUGE-L metrics, respectively, achieving a state-of-the-art result for automated bug report summarization.

Keywords

• Bug report summarization
• efficient fine-tuning
• software maintenance
• contrastive representation