Cooperative Adaptive Cruise Control Based on Reinforcement Learning for Heavy-Duty BEVs

Abstract

This study proposes a novel approach for cooperative adaptive cruise control (CACC) based on the twin delayed deep deterministic policy gradient algorithm (TD3) for heavy duty battery electric vehicles (BEVs). CACC is an advanced driver assistance systems (ADAS) that exploits vehicle connectivity to bring new advantages to cruise control technologies. The TD3 algorithm, which is a deep reinforcement learning (DRL) algorithm, was selected because it is currently at the forefront of the state of the art for problems with continuous states and actions. Furthermore, compared to state-of-the-art techniques, such as linear MPC, a DRL approach is more effective in dealing with highly nonlinear objectives. This enables us to explicitly model the effect of air drag reduction in the ego vehicle, which positively affects energy savings. The air drag reduction characteristic was modeled through experimental data from a previous work. At the same time, driving comfort was also optimized with respect to the reference driving cycle, chosen as the HHDDT driving cycle. Three different types of spacing strategies have been investigated that involve minimum time headway and time-to-collision (TTC) to study the safety guarantee of the algorithm, particularly when facing critical and unexpected situations such as sudden hard braking. The results achieved show how the Ego vehicle can reduce energy consumption by up to 19.8% without the comfort worsening with respect to the preceding vehicle, still guaranteeing safe driving conditions, when considering the spacing strategy based only on TTC, developed to obtain the highest air drag reduction.

Keywords

• Cooperative adaptive cruise control
• reinforcement learning
• TD3 algorithm
• heavy-duty vehicle
• air drag reduction
• BEV