A deep reinforcement learning with dynamic spatio-temporal graph model for solving urban logistics delivery planning problems

Abstract

The urban logistics delivery planning problems are a crucial component of urban spatial decision analysis. Most studies typically focus on traditional urban logistics delivery planning problems and ignore real-time traffic information. With the advancement of urbanization, real-time traffic networks play a critical role. However, previous studies have utilized heuristic methods to solve urban logistics delivery planning with real-time traffic information problems, and few studies have applied deep reinforcement learning methods to tackle this problem. Deep reinforcement learning methods solving traditional logistics delivery planning problems overlook the impact of dynamic spatio-temporal features on route planning. In this study, we propose a new deep reinforcement method called DRLDSTG. The method introduces the dynamic spatio-temporal graph model into a deep reinforcement learning method to capture these dynamic features from urban logistics delivery planning tasks. The actor-critic with maximum entropy method is employed to train the model and determine the optimal policy function. The experimental results indicated that the proposed method can achieve a superior solution with faster computational efficiency compared to commercial software and heuristic methods. Compared to other deep reinforcement learning methods, our method can more effectively learn dynamic spatio-temporal features from environments, demonstrating promising applications in cities.

Keywords

• Urban spatial decision analysis
• urban logistics delivery planning problems
• real-time traffic
• deep reinforcement learning
• graph model