Clustering-based label estimation for network anomaly detection

Abstract

A substantial body of work has been done to identify network anomalies using supervised and unsupervised learning techniques with their unique strengths and weaknesses. In this work, we propose a new approach that takes advantage of both worlds of unsupervised and supervised learnings. The main objective of the proposed approach is to enable supervised anomaly detection without the provision of the associated labels by users. To this end, we estimate the labels of each connection in the training phase using clustering. The “estimated” labels are then utilized to establish a supervised learning model for the subsequent classification of connections in the testing stage. We set up a new property that defines anomalies in the context of network anomaly detection to improve the quality of estimated labels. Through our extensive experiments with a public dataset (NSL-KDD), we will prove that the proposed method can achieve performance comparable to one with the “original” labels provided in the dataset. We also introduce two heuristic functions that minimize the impact of the randomness of clustering to improve the overall quality of the estimated labels.

Keywords

• Label estimation
• Network anomaly detection
• Clustering randomness