A Multiscale Interactive Attention Short Text Classification Model Based on BERT

Abstract

Text classification tasks aim to comprehend and classify text content into specific classifications. This task is crucial for interpreting unstructured text, making it a foundational task in the field of Natural Language Processing(NLP). Despite advancements in large language models, lightweight text classification via these models still demands substantial computational resources. Therefore, this paper presents a multiscale interactive attention short text classification model based on BERT, which is designed to address the short text classification problem with limited resources. A corpus containing news articles, Chinese comments, and English sentiment classifications is employed for text classification. The model uses BERT pre-trained word vectors as embedding layers, connects to a multilevel feature extraction network, and further extracts contextual features after feature fusion. The experimental results on the THUCNews, Today’s headline news corpus, the SST-2 dataset, and the Touhou 38 W dataset demonstrate that our method outperforms all existing algorithms in the literature.

Keywords

• BERT
• RNN
• CNN
• multiscale interactive attention
• pre-training models