Multi-objective virtual network embedding algorithm based on Q-learning and curiosity-driven

Abstract

Abstract Network virtualization is a vital technology that helps overcome shortcomings such as network ossification of the current Internet architecture. However, virtual network embedding (VNE) involving the allocation of resources for heterogeneous virtual network requests (VNRs) on the substrate network (SN) is considered as NP-hard problem. VNE process may involve conflicting objectives, including energy saving and VNR acceptance rate as the most critical. In this paper, we propose a virtual network multi-objective embedding algorithm based on Q-learning and curiosity-driven (Q-CD-VNE) for improving the performance of the system by optimizing conflicting objectives, namely energy saving and acceptance rate. The proposed algorithm employs Q-learning and curiosity-driven mechanism by considering other non-deterministic factors to avoid falling into a local optimum. The major contributions of this work involve (1) modeling of the multi-objective deterministic factors as binary (0, 1) integer programming problem, (2) formulating the virtual node mapping problem using the Markov decision process (MDP), (3) solving the VNE problem using Q-learning algorithm, (4) mining non-deterministic factors using curiosity-driven mechanism for avoiding prematurely falling into the Exploration-Exploitation dilemma and local optimal. Experimental results in comparison with representative researches in the field prove that the proposed algorithm can reduce energy consumption, improve the request acceptance rate, and improve the long-term average income.

Keywords

• Network virtualization
• Q-learning
• Curiosity-driven
• Energy-aware