Scalable and Autonomous Network Defense Using Reinforcement Learning

Abstract

An autonomous network defense method under attack is a critical part of preventing network infrastructure from potential damage in real time. Despite various network intrusion detection techniques, our network space is not safe enough due to the increasing exploitation of software vulnerabilities. Thus, timely response and defense methods under network intrusion are important techniques given the large scope of cyberattacks in recent years. In this paper, we design a scalable and autonomous network defense method by using the model of a zero-sum Markov game between an attacker and a defender agent. To scale up the proposed defense model, we utilize a graph convolutional network (GCN) along with framestacking to address the partial observability of the environment. The agents are trained using Proximal Policy Optimization (PPO) which allows for good convergence in a reasonable timeframe. In experiments, we evaluate the proposed model under the large network size while simulating network dynamics including link failures and other network events. The experimental results demonstrate that the proposed method scales well for larger networks and achieves state of the art results on various threat scenarios.

Keywords

• Reinforcement learning
• network defense
• Markov games
• deep learning
• graph convolutional networks