Sentiment Analysis Based on Improved Transformer Model and Conditional Random Fields

Abstract

With the rapid development of the Internet, people independently write comments with emotional characteristics on e-commerce platforms, which express consumers’ emotional tendencies towards products or services from multiple perspectives. The sentiment analysis technology of product reviews has attracted more and more attention. In recent years, the Transformer model has performed well in the field of text sentiment analysis. However, in the process of text emotion classification, the Transformer basic model cannot be well obtained when the distance between words is relatively long, and there are problems such as a low accuracy rate and recall rate and too much time spent on the overall training of model construction. Using conditional random field (CRF) as a classifier can effectively solve the problem of too long text segmentation distance, reduce model training time, and increase the accuracy of text sentiment analysis. Therefore, this paper proposes a new method for text sentiment analysis combining the improved Transformer model and a conditional random field. First of all, this paper enhances the Transformer model by adapting the decoder to better suit the task of sentiment classification, and introduces an enhanced version of the Transformer model. It is then classified by combining long-short-term memory (LSTM) and CRF. The experimental results show that in the IMDB data set, the Transformer CRF model has higher accuracy, recall rate, and F1 value, reaching 85.51%, 83.77%, and 85.06%, respectively. Compared with other methods, the results of evaluation indicators further verify that the text method has better recognition performance and generalization ability, and at the same time, it has a better understanding of customers’ emotions towards a specific aspect, provides users with accurate services, effectively improves user satisfaction, and has a high use value for enterprises’ business decisions.

Keywords

• Transformer model
• conditional random field
• sentiment analysis
• deep learning
• LSTM
• attention mechanism