Sentiment Analysis of Review Text Based on BiGRU-Attention and Hybrid CNN

Abstract

Convolutional neural networks (CNN), recurrent neural networks (RNN), attention, and their variants are extensively applied in the sentiment analysis, and the effect of fusion model is expected to be better. However, fusion model is confronted with some problems such as complicated structure, excessive trainable parameters, and long training time. The classification effect of traditional model with cross entropy loss as loss function is undesirable since sample category imbalance as well as ease and difficulty of sample classification is not taken into account. In order to solve these problems, the model BiGRU-Att-HCNN is proposed on the basis of bidirectional gated recurrent unit (BiGRU), attention, and hybrid convolutional neural networks. In this model, BiGRU and self-attention are combined to acquire global information, and key information weight is supplemented. Two parallel convolutions (dilated convolution and standard convolution) are used to obtain multi-scale characteristic information with relatively less parameters, and the standard convolution is replaced with depthwise separable convolution with two-step calculations. Traditional max-pooling and average-pooling are discarded, and global average pooling is applied to substitute the pooling layer and the fully-connected layer simultaneously, making it possible to substantially decrease the number of model parameters and reduce over-fitting. In our model, focal loss is used as the loss function to tackle the problems of unbalanced sample categories and hard samples. Experimental results illustrate that in terms of multiple indicators, our model outperforms the 15 benchmark models, even with intermediate number of trainable parameters.

Keywords

• Bidirectional gated recurrent unit
• depthwise separable convolution
• dilated convolution
• focal loss
• self-attention mechanism