Cooperative Merging Control Based on Reinforcement Learning With Dynamic Waypoint

Abstract

Reinforcement learning algorithms can cooperate with trajectory planning idea to improve the training efficiency in the field of autonomous driving for the fixed geometric constraints of the road and limited dynamics. In this study, we propose a Dynamic Waypoint Proximal Policy Optimization (DW-PPO) framework for the merging into a platoon scenario, in which the target location is constantly changing as the platoon travels. Specifically, we set up a waypoint generator based on Bezier curve to aid in the composition of the state space and reward calculation. Moreover, we refine the waypoint tracking reward in terms of both distance and direction and add an additional merging reward to complete the merging task. We test our model on three dimensions: learning performance, control performance, and generalization performance and compare with baseline model. Experimental results show that our proposed method has better training efficiency, control stability and generalization ability.

Keywords

• Autonomous driving
• deep reinforcement learning
• merging control
• cooperative driving
• proximal policy optimization