Similarity Based Feature Transformation for Network Anomaly Detection

Abstract

The fundamental objective behind any network intrusion detection system is to automate the detection process whenever intrusions occur in the network. The problem of the network anomaly detection is to determine, if the network incoming traffic is legitimate (or) anomalous. Automated detection systems designed to identify incoming anomalous traffic patterns usually apply widely used machine learning techniques. However, irrespective of any system model which is developed to identify anomalous traffic, all these models requires comparing anomalous and normal traffic patterns. Such comparisons implicitly depend on the ability of the underlying machine learning model to gauge the similarity between a known legitimate observation and the target. The efficiency of any network anomalous detection system depends on the use of distance (or) similarity measures and how they are actually applied. A novel distance function which can be applied to determine the similarity between two conditional feature pattern vectors is an important contribution of present research. Feature dimensionality is another important issue for any machine learning algorithm. In the present work, feature reduction is achieved using the proposed feature transformation technique. However, our approach for feature transformation uses the proposed gaussian distance function to achieve dimensionality reduction to represent the original input dataset in the new transformation space. We have also proposed new computation expressions for determining equivalent deviation and threshold in gaussian space. Experiments are performed on KDD and NSL-KDD datasets by considering widely applied classifier algorithms in various state-of-art research contributions. For performance validation of machine learning models, k-fold cross validation is applied by setting k to 10 through considering evaluation parameters such as accuracy, precision and recall. Experiment results have proved that our approach for anomaly detection that applies the proposed feature transformation technique proved comparatively better to detection methods CANN, GARUDA, and UTTAMA addressed in the recent research literature.

Keywords

• Similarity function
• feature clustering
• intrusion
• conditional feature pattern vector
• anomaly detection