PDHS: Pattern-Based Deep Hate Speech Detection With Improved Tweet Representation

Abstract

Automatic hate speech identification in unstructured Twitter is significantly more difficult to analyze, posing a significant challenge. Existing models heavily depend on feature engineering, which increases the time complexity of detecting hate speech. This work aims to classify and detect hate speech using a linguistic pattern-based approach as pre-trained transformer language models. As a result, a novel Pattern-based Deep Hate Speech (PDHS) detection model was proposed to detect the presence of hate speech using a cross-attention encoder with a dual-level attention mechanism. Instead of concatenating the features, our model computes dot product attention for better representation by reducing the irrelevant features. The first level of Attention is extracting aspect terms using predefined parts-of-speech tagging. The second level of Attention is extracting the sentiment polarity to form a pattern. Our proposed model trains the extracted patterns with term frequency, parts-of-speech tag, and Sentiment Scores. The experimental results on Twitter Dataset can learn effective features to enhance the performance with minimum training time and attained 88%F1Score.

Keywords

• Attention mechanism
• transformer
• BERT
• sequence modeling
• hate speech
• natural language processing