Enhancing Longitudinal Velocity Control With Attention Mechanism-Based Deep Deterministic Policy Gradient (DDPG) for Safety and Comfort

Abstract

The transportation system is moving towards higher levels of autonomy. Longitudinal speed control, or adaptive cruise control (ACC), is intended to provide driver assistance by controlling vehicle speed and maintaining a safe distance from the preceding vehicle. Artificial intelligence (AI) and machine learning (ML) paved the way for robust navigation and decision-making in complex environments. In this paper, we propose a comprehensive framework for speed control using a deep reinforcement learning algorithm for safety and comfort. We incorporate the deep deterministic policy gradient (DDPG) framework with an attention mechanism for this. Many works achieved smooth vehicle control output from the DDPG algorithm with different network structures, but we demonstrate that adding an attention mechanism improves the overall performance. The baseline DDPG framework is based on fully connected layers. However, when we bring the attention mechanism to the DDPG model, it helps to increase focus on the most important features and enhances the overall model effectiveness. We also designed a custom reward function as our priority is to improve overall safety and comfort. The efficacy of our model is evaluated in comparison to the baseline DDPG model, emphasizing the influence of the attention mechanism. We have performed an ablation study to determine the impact of the number of layers and neurons in the hidden layer. In addition, we tested this model on diverse datasets (simulated and real), including some unknown scenarios in the testing datasets. We demonstrate that our architecture has outperformed the state-of-the-art model, and is quite robust across different datasets.

Keywords

• ACC
• attention mechanism
• DDPG
• machine learning
• reinforcement learning
• longitudinal velocity control