InDS: Intelligent DRL Strategy for Effective Virtual Network Embedding of an Online Virtual Network Requests

Abstract

Network virtualization is a demanding feature in the evolution of future Internet architectures. It enables on-demand virtualized resource provision for heterogeneous Virtual Network Requests (VNRs) from diverse end users over the underlying substrate network. However, network virtualization provides various benefits such as service separation, improved Quality of Service, security, and more prominent resource usage. It also introduces significant research challenges. One of the major such issues is allocating substrate network resources to VNR components such as virtual machines and virtual links, also named as the virtual network embedding, and it is proven to be $\mathbb {N}\mathbb {P}$ -hard. To address the virtual network embedding problem, most of the existing works are 1) Single-objective, 2) They failed to address dynamic and time-varying network states 3) They neglected network-specific features. All these limitations hinder the performance of existing approaches. This work introduces an embedding framework called Intelligent Deep Reinforcement Learning (DRL) Strategy for effective virtual network embedding of an online VNRs (InDS). The proposed InDS uses an actor-critic model based on DRL architecture and Graph Convolutional Networks (GCNs). The GCN effectively captures dependencies between the VNRs and substrate network environment nodes by extracting both network and system-specific features. In DRL, the asynchronous advantage actor-critic agents can learn policies from these features during the training to decide which virtual machines to embed on which servers over time. The actor-critic helps in efficiently learning optimal policies in complex environments. The suggested reward function considers multiple objectives and guides the learning process effectively. Evaluation of simulation results shows the effectiveness of InDS in achieving optimal resource allocation and addressing diverse objectives, including minimizing congestion, maximizing acceptance, and revenue-to-cost ratios. The performance of InDS exhibits superiority in achieving 28% of the acceptance ratio and 45% of the revenue-to-cost ratio by effectively managing the network congestion compared to other existing baseline works.

Keywords

• Network virtualization
• deep reinforcement learning
• resource utilization
• network features
• congestion
• acceptance ratio