An Intelligent SDWN Routing Algorithm Based on Network Situational Awareness and Deep Reinforcement Learning

Abstract

To address the challenges of obtaining network state information, flexibly forwarding data, and improving the communication quality of service (QoS) in wireless network transmission environments in response to dynamic changes in network topology, this paper introduces an intelligent routing algorithm based on deep reinforcement learning (DRL) with network situational awareness under a software-defined wireless networking (SDWN) architecture. First, comprehensive network traffic information is collected under the SDWN architecture, and a graph convolutional network-gated recurrent unit (GCN-GRU) prediction mechanism is used to perceive future traffic trends. Second, a proximal policy optimization (PPO) DRL-based data forwarding mechanism is designed in the knowledge plane. The predicted network traffic matrix and topology information matrix are treated as the DRL environment, while next-hop adjacent nodes are treated as executable actions, and action selection policies are designed for different network conditions. To guide the learning and improvement of the DRL agent’s routing strategy, reward functions of different forms are designed by utilizing network link information and different penalty mechanisms. Additionally, importance sampling steps and gradient clipping methods are employed during gradient updating to enhance the convergence speed and stability of the designed intelligent routing method. Experimental results show that this solution outperforms traditional routing methods in network throughput, delay, packet loss rate, and wireless node distance. Compared to value-function-based Dueling Deep Q-Network (DQN) routing, the convergence of the proposed method is significantly faster and more stable. Simultaneously, hardware storage consumption is reduced, and real-time routing decisions can be made using the current network state information. The source code can be accessed at https://github.com/GuetYe/DRL-PPONSA.

Keywords

• Deep reinforcement learning
• gradient clipping
• intelligent routing
• importance sampling
• network situational awareness
• software-defined wireless networking