Autonomous Federated Learning for Distributed Intrusion Detection Systems in Public Networks

Abstract

The rapid integration of IoT, cloud, and edge computing has resulted in highly interconnected networks, emphasizing the need for advanced Intrusion Detection Systems (IDS) to maintain security. Successful AI-based IDS relies on high-quality data for model training. Even though a vast array of datasets from controlled settings are accessible, many fall short as they are outdated and lack the representative data of network traffic dynamics typically seen in public networks. This paper aims to advance understanding in designing testbed architectures for defense mechanisms within public networks. At its core, this research introduces a unique testbed utilizing the connectivity of panOULU Municipal public network in the city of Oulu, Finland. This experimental setup examines AI-driven security across the public network. It utilizes edge-to-cloud infrastructures, incorporating Software-Defined Networking (SDN) and Network Function Virtualization (NFV) via the VMware vSphere platform. During the training phase, a script distinguishes incoming packets as either benign or malicious based on well-defined local parameters and simulated attack scenarios. This labeled data is then utilized for training machine learning models within the Federated Learning framework, FED-ML. Subsequently, these models are evaluated on previously unseen data. The entire procedure, from traffic gathering to model training, operates without human involvement. The evaluation dataset and testbed configuration we have made publicly available through this research can deepen our understanding of the challenges in safeguarding public networks, especially those that blend various technologies in diverse environments.

Keywords

• Network security
• cybersecurity
• federated learning
• data engineering
• distributed computing
• stream processing