Rule-Guidance Reinforcement Learning for Lane Change Decision-making: A Risk Assessment Approach

Abstract

Abstract To solve problems of poor security guarantee and insufficient training efficiency in the conventional reinforcement learning methods for decision-making, this study proposes a hybrid framework to combine deep reinforcement learning with rule-based decision-making methods. A risk assessment model for lane-change maneuvers considering uncertain predictions of surrounding vehicles is established as a safety filter to improve learning efficiency while correcting dangerous actions for safety enhancement. On this basis, a Risk-fused DDQN is constructed utilizing the model-based risk assessment and supervision mechanism. The proposed reinforcement learning algorithm sets up a separate experience buffer for dangerous trials and punishes such actions, which is shown to improve the sampling efficiency and training outcomes. Compared with conventional DDQN methods, the proposed algorithm improves the convergence value of cumulated reward by 7.6% and 2.2% in the two constructed scenarios in the simulation study and reduces the number of training episodes by 52.2% and 66.8% respectively. The success rate of lane change is improved by 57.3% while the time headway is increased at least by 16.5% in real vehicle tests, which confirms the higher training efficiency, scenario adaptability, and security of the proposed Risk-fused DDQN.

Keywords

• Autonomous driving
• Reinforcement learning
• Decision-making
• Risk assessment
• Safety filter