Enhanced Transformer-BLSTM Model for Classifying Sentiment of User Comments on Movies and Books

Abstract

Classifying the sentiment of user comments on a website is a crucial task within Natural Language Processing (NLP). Conducting sentiment analysis can aid businesses in gaining a more profound comprehension and examination of users’ emotional inclinations towards products or services. This study introduces a sentiment classification model that combines the Transformer and BLSTM architectures to analyze the sentiment of user comments on movie and book websites. By incorporating the strengths of both Transformer and BLSTM, the proposed model mitigates the issue of vanishing gradient by scrutinizing inputs within a long-term context using BLSTM. It employs the multi-head attention mechanism of the Transformer to extract features and capture significant semantic details within the comments. Furthermore, the joint model combines the TF-IDF weights with the vector space, which improves the embedding process. The proposed model’s effectiveness was evaluated by categorizing the sentiment of user comments on publicly available datasets containing more than 20,000 movie and book comments. The results indicate that the proposed model is superior to LSTM and CNN in sentiment classification tasks. Moreover, the proposed approach has demonstrated significant improvements, particularly in the training set, achieving an accuracy of 93.81%.

Keywords

• NLP
• sentiment analysis
• transformer
• BLSTM
• TF-IDF