Anomaly Detection Framework of System Call Trace Based on Sequence and Frequency Patterns

Abstract

The existing system call-based anomaly intrusion detection methods can’t accurately describe the behavior of the process by a single trace pattern.In this paper,the process behavior is modeled based on the sequence and frequency patterns of system call trace,and a data-driven anomaly detection framework is designed.The framework could detect both sequential and quantitative anomalies of the system call trace simultaneously.With the help of combinational window mechanism,the framework could realize offline fine-grained learning and online anomaly real-time detection by meeting different requirements of offline trai-ning and online detection for extracting trace information.Performance comparison experiments of unknown anomalies detection are conducted on the ADFA-LD intrusion detection standard dataset.The results show that,compared with the four traditional machine learning methods and four deep learning methods,the comprehensive detection performance of the framework improves by about 10%.

Keywords

• host-based intrusion detection systems|system calls|deep neural network|long and short-term memory neural network