Aspect-Level Sentiment Analysis Using CNN Over BERT-GCN

Abstract

Context-based GCNs have achieved relatively good effectiveness in the sentiment analysis task, especially aspect-level sentiment analysis (ALSA). However, the previous context-based GCNs for ALSA often used GCNs with the following limitations: (i) Using GCNs limited to a few layers (two or three) due to the vanishing gradient, limiting their performance. (ii) Not considering helpful information about the hidden context between the words. To solve these limitations, this paper proposes a novel CNN over the BERT-GCN model for ALSA. The contributions of the proposed method are summarized as follows: (i) Handling the disadvantage of limiting the GCN to a few layers by adding convolutional layers of the convolutional neural network (CNN) model after GCN layers. (ii) Considering further helpful information about the hidden context between the words by combining the Bidirectional Encoder Representations from Transformers (BERT) and Bidirectional Long Short Term Memory (BiLSTM) models. The proposed model includes the following steps: First, words in sentences are converted vectors using BERT. Second, the contextualized word representations are created based on BiLSTM over word vectors. Third, significant features are extracted and represented using the GCN model with multiple convolutional layers over the contextualized word representations. Finally, the aspect-level sentiments are classified using the CNN model over the feature vectors. Experiments on three benchmark datasets illustrate that our proposed model has improved the performance of the previous context-based GCN methods for ALSA.

Keywords

• Aspect-level sentiment analysis
• GCN
• CNN
• BiLSTM
• BERT-GCN