A Parameter Space Framework for Online Outlier Detection Over High-Volume Data Streams

Abstract

In diverse applications ranging from social networks to location-based online services to traffic monitoring, data streams are continuously monitored by multiple outlier analysts customized with different parameter settings. Real-time response to such complex outlier analytics in high-speed streaming data has been recognized as critical for many domains. In this paper, we propose a parameter space framework, called PSOD, for online outlier detection over sliding window streams to support a large variety of query requests in parameter space with both diverse pattern and window parameter settings. First, we design an ingenious neighbor table that records the neighbors for each point in different distance intervals and different slides, which enables us to maximally reuse the already acquired neighbor information across the entire parameter space. In addition, we propose a series of shared strategies in sliding window environment to minimize processing cost by eliminating the redundant query requests. Moreover, the PSOD effectively transforms the query group in 4-D parameter space into a periodic query group in 3-D parameter space to minimize the number of queries. Our experimental study on three real-world steaming data demonstrates that our PSOD successfully drives down the CPU costs by more than 100 folds compared with the state-of-the-art method.

Keywords

• Outlier detection
• data streams
• multi-query
• parameter space