Semi-Supervised Encrypted Malicious Traffic Detection Based on Multimodal Traffic Characteristics

Abstract

The exponential growth of encrypted network traffic poses significant challenges for detecting malicious activities online. The scale of emerging malicious traffic is significantly smaller than that of normal traffic, and the imbalanced data distribution poses challenges for detection. However, most existing methods rely on single-category features for classification, which struggle to detect covert malicious traffic behaviors. In this paper, we introduce a novel semi-supervised approach to identify malicious traffic by leveraging multimodal traffic characteristics. By integrating the sequence and topological information inherent in the traffic, we achieve a multifaceted representation of encrypted traffic. We design two independent neural networks to learn the corresponding sequence and topological features from the traffic. This dual-feature extraction enhances the model’s robustness in detecting anomalies within encrypted traffic. The model is trained using a joint strategy that minimizes both the reconstruction error from the autoencoder and the classification loss, allowing it to effectively utilize limited labeled data alongside a large amount of unlabeled data. A confidence-estimation module enhances the classifier’s ability to detect unknown attacks. Finally, our method is evaluated on two benchmark datasets, UNSW-NB15 and CICIDS2017, under various scenarios, including different training set label ratios and the presence of unknown attacks. Our model outperforms other models by 3.49% and 5.69% in F1 score at labeling rates of 1% and 0.1%, respectively.

Keywords

• encrypted malicious traffic detection
• semi-supervised learning
• multimodal features
• network security