A Neural Network-Based Approach for Cryptographic Function Detection in Malware

Abstract

Cryptographic technology has been commonly used in malware for hiding their static characteristics and malicious behaviors to avoid the detection of anti-virus engines and counter the reverse analysis from security researchers. The detection of cryptographic functions in an effective way in malware has vital significance for malicious code detection and deep analysis. Many efforts have been made to solve this issue, while existing methods suffer from some issues, such as unable to achieve promising results in accuracy, limited by prior knowledge, and have a high overhead. In this paper, we draw on the idea of text classification in the field of natural language processing and propose a novel neural network to detect the type of cryptographic functions. The new network is an end-2-end model which includes two important modules: Instruction-2-vec and K-Max-CNN-Attention. The Instruction-2-vec model extracts the “words” of assembly instructions and transfers them into continuous vectors. The K-Max-CNN-Attention is used to encode the instruction vectors and generate the representation of the function. And we designed a softmax classifier to predict the categories of the functions. Extensive experiments were conducted on a collected dataset which contains 15 common types of cryptographic functions extracted from malware, to assess the validity of the proposed approach. The experiment results showed that the proposed approach archives a better performance than the recent embedding network SAFE with the Precision, Recall and F1-score of 0.9349, 0.8933 and 0.9020, respectively. We also compared it with four widely-used tools, the results demonstrated that our approach is much better in accuracy and effectiveness than all of them.

Keywords

• Cryptographic function detection
• neural network
• function embedding
• binary analysis