Blin: A Multi-Task Sequence Recommendation Based on Bidirectional KL-Divergence and Linear Attention

Abstract

Sequence recommendation is a prominent research area within recommender systems, focused on predicting items that users may be interested in by modeling their historical interaction sequences. However, due to data sparsity, user interaction sequences in sequence recommendation are typically short. A common approach to address this issue is filling sequences with zero values, significantly reducing the effective utilization of input space. Furthermore, traditional sequence recommendation methods based on self-attention mechanisms exhibit quadratic complexity with respect to sequence length. These issues affect the performance of recommendation algorithms. To tackle these challenges, we propose a multi-task sequence recommendation model, Blin, which integrates bidirectional KL divergence and linear attention. Blin abandons the conventional zero-padding strategy, opting instead for random repeat padding to enhance sequence data. Additionally, bidirectional KL divergence loss is introduced as an auxiliary task to regularize the probability distributions obtained from different sequence representations. To improve the computational efficiency compared to traditional attention mechanisms, a linear attention mechanism is employed during sequence encoding, significantly reducing the computational complexity while preserving the learning capacity of traditional attention. Experimental results on multiple public datasets demonstrate the effectiveness of the proposed model.

Keywords

• sequence recommendation
• self-attention mechanism
• consistency training
• data augmentation