Research on Integrated Control Strategy for Highway Merging Bottlenecks Based on Collaborative Multi-Agent Reinforcement Learning

Abstract

The merging behavior of vehicles at entry ramps and the speed differences between ramps and mainline traffic cause merging traffic bottlenecks. Current research, primarily focusing on single traffic control strategies, fails to achieve the desired outcomes. To address this issue, this paper explores an integrated control strategy combining Variable Speed Limits (VSL) and Lane Change Control (LCC) to optimize traffic efficiency in ramp merging areas. For scenarios involving multiple ramp merges, a multi-agent reinforcement learning approach is introduced to optimize control strategies in these areas. An integrated control system based on the Factored Multi-Agent Centralized Policy Gradients (FACMAC) algorithm is developed. By transforming the control framework into a Decentralized Partially Observable Markov Decision Process (Dec-POMDP), state and action spaces for heterogeneous agents are designed. These agents dynamically adjust control strategies and control area lengths based on real-time traffic conditions, adapting to the changing traffic environment. The proposed Factored Multi-Agent Centralized Policy Gradients for Integrated Traffic Control in Dynamic Areas (FM-ITC-Darea) control strategy is simulated and tested on a multi-ramp scenario built on a multi-lane Cell Transmission Model (CTM) simulation platform. Comparisons are made with no control and Factored Multi-Agent Centralized Policy Gradients for Integrated Traffic Control (FM-ITC) strategies, demonstrating the effectiveness of the proposed integrated control strategy in alleviating highway ramp merging bottlenecks.

Keywords

• variable speed limits
• lane change control
• multi-agent reinforcement learning
• highway merging bottlenecks
• integrated control strategy