TransLSTM: A hybrid LSTM-Transformer model for fine-grained suggestion mining

Abstract

Digital platforms on the internet are invaluable for collecting user feedback, suggestions, and opinions about various topics, such as company products and services. This data is instrumental in shaping business strategies, enhancing product development, and refining service delivery. Suggestion mining is a key task in natural language processing, which focuses on extracting and analysing suggestions from these digital sources. Initially, suggestion mining utilized manually crafted features, but recent advancements have highlighted the efficacy of deep learning models, which automatically learn features. Models like Convolutional Neural Networks (CNN), Recurrent Neural Networks (RNN), Long Short-Term Memory (LSTM), and Bidirectional Encoder Representations from Transformers (BERT) have been employed in this field. However, considering the relatively small datasets and the faster training time of LSTM compared to BERT, we introduce TransLSTM, a novel LSTM-Transformer hybrid model for suggestion mining. This model aims to automatically pinpoint and extract suggestions by harnessing both local and global text dependencies. It combines the sequential dependency handling of LSTM with the contextual interaction capabilities of the Transformer, thus effectively identifying and extracting suggestions. We evaluated our method against state-of-the-art approaches using the SemEval Task-9 dataset, a benchmark for suggestion mining. Our model shows promising performance, surpassing existing deep learning methods by 6.76% with an F1 score of 0.834 for SubTask A and 0.881 for SubTask B. Additionally, our paper presents an exhaustive literature review on suggestion mining from digital platforms, covering both traditional and state-of-the-art text classification techniques.

Keywords

• Suggestion mining
• Long Short-Term Memory (LSTM) network
• Switch transformer
• Digital platforms