Detection and diagnosis of unknown threats in power equipment using machine learning and Spark technology

Abstract

With the continuous advancement of network technology, attack behaviors have become increasingly diversified, giving rise to new challenges in threat detection. To effectively monitor and diagnose unknown threats, we have created an unknown threat detection model for power equipment based on Spark technology. Our research utilizes a lightweight gradient-based method for detecting known threats, and we propose a novel detection approach for unknown threats that combines classical anomaly detection methods, specifically support vector machines, with autoencoders. In addition, Spark technology is employed to achieve data parallelization, enhancing detection and diagnosis efficiency. Finally, we apply stacking techniques to integrate the two detection methods, enabling hybrid intrusion detection and diagnosis. Experimental analysis indicates that the model runs in 1.88 seconds, achieving a detection accuracy of 98.88%, a precision rate of 99.06%, and a false positive rate of 2.36%. This approach allows for more efficient and accurate detection of unknown threat attacks on power grid equipment, providing robust network security for power systems. Our findings offer a new theoretical perspective for the evolving field of network security.