Locality and Consistency Based Sequential Ensemble Method for Outlier Detection

Abstract

Outlier detection has been widely used in many fields,such as network intrusion detection,credit card fraud detection,etc.The increase in data dimensions leads to many irrelevant and redundant features,which will obscure the relevant features and result in false positive results.Due to the sparseness and distance aggregation effects of high-dimensional data,the traditional outlier detection algorithms based on density and distance are no longer applicable.Most of the outlier detection research based on machine learning focuses on a single model,which has certain deficiencies in anti-overfitting ability.The ensemble learning model has good generalization ability,and in actual application shows better prediction accuracy than the single model.This paper proposes an outlier detection sequence integration method LCSE based on neighborhood consistency (locality and consistency based sequential ensemble method for outlier detection).Firstly,it constructs a basic model of outlier detection based on diversity,secondly,selects the abnormal candidate points according to the global integration consistency,and finally considers the local neighborhood correlation of the data to select and combine the basic model results.Experiments verify that LCSE has an average outlier detection accuracy increase of 20.7% compared with traditional methods.Compared with the ensemble methods LSCP_AOM and iForest,the performance is increased by 3.6% on average.Therefore,it is better than other ensemble methods and neural network methods.

Keywords

• high-dimensional data|outlier detection|ensemble diversity|ensemble consistency|neighborhood correlation