Model-Based Graph Reinforcement Learning for Inductive Traffic Signal Control

Abstract

We introduce MuJAM, an adaptive traffic signal control method which leverages model-based reinforcement learning to 1) extend recent generalization efforts (to road network architectures and traffic distributions) further by allowing a generalization to the controllers’ constraints (cyclic and acyclic policies), 2) improve performance and data efficiency over related model-free approaches, and 3) enable explicit coordination at scale for the first time. In a zero-shot transfer setting involving both road networks and traffic settings never experienced during training, and in a larger transfer experiment involving the control of 3,971 traffic signal controllers in Manhattan, we show that MuJAM, using both cyclic and acyclic constraints, outperforms domain-specific baselines as well as a recent transferable approach.

Keywords

• Adaptive traffic signal control
• transfer learning
• multi-agent reinforcement learning
• joint action modeling
• model-based reinforcement learning
• graph neural networks