ESTIMATION OF MALWARE DETECTION ALGORITHM ACCURACY BASED ON ANOMALY SEARCH IN PROGRAM BEHAVIOR

Abstract

Subject of Research.The paper deals with the algorithm of anomaly detection in the behavior of operating system processes caused by the execution of previously unknown parts of the program code. The algorithm is implemented in the novel intrusion detection system CODA. A testing algorithm allows reducing test time and increasing its accuracy. Method. The proposed detection method is based on creation ofbehavior model for legitimate process using sequences of system calls. Measures of similarity between an arbitrary process and a model are proposed. They allow interpreting the problem of anomaly detection as the problem of vector classification. In order to evaluate the accuracy of the anomaly detection algorithm, the accuracy of the classifier is proposed to be evaluated by cross-validation method. Neural network of perceptron type was used as a classifier. Main Results.A platform for the mass distributed testing of malicious programs in virtual machines was developed. Open source library for distributed computing BOINC was used in the platformimplementation. Academic base of malware and open base Malwr was used to select 60 thousand malicious programs. From the general base33.13% of malware have workedcorrectly. A model of legitimate processes runningwithin half an hourwas created. Estimates ofmalware behavior were recorded as vectors. The most accurate neural network was searched for these vectors classification. Neural networks with different teaching parameters and different number of neurons in a hidden layer were looked over. The most precise perceptron was discovered. The accuracy of the best classifier was 91%. Practical Relevance. The results can be useful in malware detection. Our algorithm does not require Internet connection.It can find both old and new malware.

Keywords

• anomaly detection
• malware detection
• dynamic analysis
• behavior analysis
• neural networks