TransNet: Deep Attentional Hybrid Transformer for Arabic Posts Classification

Abstract

Sentiment analysis is important for comprehending attitudes and emotions, and popular social media platform X (formally Twitter) is useful in this context. For sentiment analysis of English texts, several approaches are available but the Arabic language calls for more specialized study because of its unique qualities and subtleties. This paper presents TransNet, a Deep Attentional Hybrid Transformer model designed to classify asthma-related Arabic social media messages. TransNet combines the sequential learning capabilities of Gated Recurrent Units (GRUs) and Long Short-Term Memory Networks (LSTMs) with the resilient attention mechanisms of Transformers. The model can extract complex patterns and relationships from textual input efficiently. We use label encoding and upsampling approaches to rectify the intrinsic class imbalance in our Kaggle dataset. We also expand the dataset containing Arabic asthma-related tweets with new data, namely by substituting synonyms to add variety to the training set. To better capture the subtleties of the Arabic language, we further improve text representation by adding a pre-trained Bidirectional Encoder Representations from Transformers (BERT) tokenizer. A rigorous collection of baseline models, including Transformer+LSTM, Transformer+GRU, Transformer+CNN, and Transformer+GRU-CNN, are used to assess TransNet’s performance. TransNet demonstrates its supremacy with an F1 score of 97.86% and an excellent accuracy of 97.87% in the empirical data. We use Local Interpretable Model-agnostic Explanations (LIME), which helps to understand the mechanism of the model and also ensures that our forecasts are clear and comprehensible. According to our study, TransNet performs better than conventional models in categorizing Arabic asthma postings on social media and gives useful insights.

Keywords

• TransNet
• hybrid transformer
• transformer
• sentiment analysis
• Arabic text classification