An Empirical Determination of Optimum Artificial Intelligence Algorithm: Detection Using Signal-to-Noise Ratio Approach

Abstract

This paper proposes a new metric to compare artificial intelligence (AI) algorithms for malware detection using system calls. With the increasing complexity of malware and the proliferation of AI algorithms, there is a need for a comprehensive metric that accounts for multiple factors and not only accuracy. Motivated by the desire to enhance algorithm selection processes, this research introduces a metric that holistically evaluates algorithm performance while considering crucial aspects such as precision and time efficiency. The metric is based on the signal-to-noise ratio (SNR), which combines multiple measures such as accuracy, precision, and time to build model. The paper shows how SNR can be calculated for different AI algorithms and system call lengths using functions of mean and variability that translate to signal and noise respectively. The paper also introduces a dataset of malicious activity for anomaly-based host-based intrusion detection systems. The paper evaluates eight AI algorithms for eleven datasets and validates the results using a test set. The paper finds that SNR is an effective and robust metric for identifying the best AI algorithm that achieves high accuracy and precision with reduced time. The paper suggests that SNR can be used as a general method for optimizing AI algorithm performance.

Keywords

• Signal-to-noise ratio
• accuracy
• precision
• time to build
• system calls