Transferability Evaluation in Wi-Fi Intrusion Detection Systems Through Machine Learning and Deep Learning Approaches

Abstract

Intrusion Detection System (IDS) plays a pivotal role in safeguarding network security. The efficacy of these systems is rigorously assessed through established metrics including precision, recall, F1 score, and AUC score. When subjected to rigorous testing on well-known datasets like AWID and AWID3, individual IDS models consistently deliver exceptional performances, boasting F1 scores ranging from 0.98 to 1 and AUC scores spanning 0.97 to 0.99. However, the true challenge surfaces when the objective is to extend the transferability of these high-performing models to entirely novel, unseen datasets. This endeavor unravels a diverse performance landscape, demonstrating that the outstanding performance observed on a particular dataset doesn’t guarantee the transferability of features across dissimilar datasets nestled within different network environments. In order to evaluate the feature transferability, we turn to AWID and AWID3 datasets as the main distinction between AWID (potentially referring to AWID2) and AWID3 lies in their specific focuses and contexts within the field of Wi-Fi intrusion detection. Although both datasets are centered on the general goal of detecting Wi-Fi intrusions, AWID3 has been carefully designed to meet the specific needs of corporate Wi-Fi applications. A comprehensive evaluation involving Multilayer Perceptron(MLP), and Convolutional Neural Networks (CNN) models has been executed, uncovering that CNN conspicuously outshines the MLP model.

Keywords

• Transferability assessment
• performance evaluation
• intrusion detection system (IDS)
• deep learning
• wireless security