An Automatic Sentiment Analysis Method for Short Texts Based on Transformer-BERT Hybrid Model

Abstract

Sentiment analysis towards short texts is always facing challenges, because short texts only contain limited semantic characteristics. As a result, this paper constructs a specific large language structure to deal with this issue. In all, a novel automatic sentiment analysis method for short texts based on Transformer-BERT hybrid model is proposed by this paper. Firstly, BERT structure is utilized to extract word vectors, and is integrated with topic vectors to improve textual feature expression ability. Then, the fused word vectors are input into a Bidirectional Gated Recurrent Unit (Bi-GRU) structure to learn contextual features. In this part, a Transformer structure is applied behind the Bi-GRU and combined with the previous module to output sentiment analysis results. In addition, Accuracy, Precision, Recall and F1 indexes were collected from real-world Twitter datasets and shopping data to evaluate the performance of the proposed method. The experimental results show that the method performs well in many indexes. Compared with traditional method, it has achieved remarkable performance improvement, and this method achieves higher accuracy and efficiency in sentiment analysis of short texts, and has good generalization ability.

Keywords

• Sentiment analysis
• short texts
• large language model
• semantic comprehension