An AI-Powered Network Threat Detection System

Abstract

The work develops a network threat detection system, AI@NTDS, that uses the behavioral features of attackers and intelligent techniques. The proposed AI@NTDS system combines data analysis, feature extraction, and feature evaluation to construct a detection model, which supports a more straightforward strategy by which the operating system or its operators can defend against network attacks. The Linux system interaction information of SSH (Secure Shell) and Telnet are obtained from the Cowrie Honeypot and labeled according to Enterprise Tactics of MITRE ATT&CK to ensure dataset credibility. The proposed AI@NTDS system has three levels, depending on the attacker’s attacks and the user’s risk of damage. Fifty-two features are used to detect the network threat level. The features contain message-based features for all kinds of Linux operating instructions, host-based features for all types of information in the network connection process, and geography-based features are related to the attacker’s location. AI-based algorithms LightGBM, Random Forest and the K-NN algorithm are used to verify the identification of the custom features. Finally, the detection model that is trained using the best combination of features is used to predict the test dataset. The accuracy of the proposed AI@NTDS system reaches 99%, 95.66%, and 94.08% with the LightGBM, Random Forest, and K-NN algorithms, respectively. The mutual dependencies of features and network threats are evaluated. Results of a performance analysis reveal that the proposed AI@NTDS system has an accuracy of 99.20% and an F1-score of 99.80%. It is superior to existing detection mechanisms, which it outperforms by 4% and 1% in accuracy and F1-score, respectively.

Keywords

• Honeypot
• intent analysis
• machine learning
• remote shell access
• threat detection