Algorithm optimization and anomaly detection simulation based on extended Jarvis-Patrick clustering and outlier detection

Abstract

In this paper, the authors analyze the algorithm optimization and anomaly detection simulation based on extended jarvis-patrick clustering and outlier detection. We perform detection by using the jarvis-patrick graph-based clustering method. After that, to further improve the false alarm rate (FAR) of the algorithm, we use an extra outlier detection step combined with our proposed EJP to create a new anomaly detection method called LD-EJP. Using LD-EJP, the false alarm rate improved much (experiments show that the false alarm rate can reach 4.1% while the best JP clustering can reach is 7.4%). Then, we tested LD-EJP against two other anomaly detection methods using k-means and LGCCB, showing that our algorithm has a better detection rate and false alarm rate than these two clustering-based anomaly detection methods. In addition, the detection rate and false positives of the algorithm also have some room for improvement. In the labeling process, the proportion of anomaly clusters to normal clusters needs to be manually adjusted to find a better detection rate. In addition, the detection rate we chose can consume some of the detection rate gained in extended JP clustering to have the LD-EJP obtain a better FAR. Therefore, our future work contains finding or proposing another outlier detection algorithm with better performance than our LD-EJP method.

Keywords

• Anomaly Detection
• Jarvis-Patrick Clustering
• Extended Shared Nearest Neighbor
• Outlier Detection