Guangtongxin yanjiu (Oct 2024)
A Reinforcement Learning based Adaptive and Efficient RWA in All Optical Networks
Abstract
【Objective】Recent research efforts on Routing and Wavelength Assignment (RWA) for all optical networks are focused on Deep Reinforcement Learning (DRL) based algorithms. The DRL based RWA algorithms are mostly rely on the K Shortest Paths (KSP) routing to calculate candidate paths in advance, hence the DRL agent can choose possible actions from the precomputed paths. These KSP based models lack of flexibility and dynamicity, since they need to re-calculate the KSP for all the node pairs once the topology changes occur. To address this issue, this paper proposes an Adaptive and Efficient(ADE)-RWA algorithm based on DRL.【Methods】The key points and innovations of the ADE-RWA lie in that during the training process, the DRL agent takes actions in a step-by-step way instead of selecting from the precomputed K complete paths. Therefore, the routing strategies are dynamically adjustable in training even under the case of topology changes. It is because that the actions are open for the agent to take without concerning the limitations of the K fixed paths. Moreover, the ADE-RWA records the successfully assigned routes during the training in a LookUp Table (LUT). The algorithm turns to LUT checking for finding the available routes once the DRL training is converged, since at that time the LUT has acquired enough information for the RWA from the DRL training. The LUT based routing can effectively reduce the computational costs and improve the efficiency of RWA. In addition, the DRL training phase and LUT routing phase are real-time switchable. The algorithm turns to the DRL training phase when a link failure caused topology change occurs, and turns back to LUT checking when the model training is converged again.【Results】Experimental results show that compared with KSP-First Fit(FF)and Deep Reinforcement Learning Framework for Routing, Modulation and Spectrum Assignment (DeepRMSA), the blocking probability of ADE-RWA is reduced by 36% and 30% respectively. When a link failure occurs, the algorithm can quickly adapt to the changes in network topology.【Conclusion】The proposed DRL based RWA framework ADE-RWA can achieve adaptive routing and wavelength allocation under dynamic network conditions with low computational cost.
