Optimizing Longitudinal Velocity Control via Self-Supervised Learning and Deep Deterministic Policy Gradient

Abstract

Artificial intelligence (AI) in autonomous vehicles (AVs) is gaining much focus on safety, comfort, and efficiency. The goal is to provide the driver with assistance and mimic human driving. Deep reinforcement learning (DRL), specifically the deep deterministic policy gradient (DDPG) is often used for longitudinal velocity control, utilizing the velocities of the vehicles and the distance between them. DDPG has been found effective in many studies when it is embedded with a self-supervised learning (SSL) method. In this paper, a framework for longitudinal velocity control is proposed using SSL and DDPG frameworks. The inputs of the DDPG networks have been replaced by the outputs of the SSL network. The primary objective of this SSL is to enable the model to accurately predict future states based on current states and actions. The input features of the datasets have been modified so that the DDPG model can get additional information that helps the model make better predictions. These features include the distance between vehicles, ego vehicle velocity, acceleration, jerk, and lead vehicle relative velocity, and estimated velocity. Furthermore, a custom reward function is designed to account for safety, driving comfort, negative impact, driving aggressiveness, and fuel efficiency. In order to evaluate the model, the algorithm has been trained and tested on a variety of datasets, including simulated and real-world data. The analysis demonstrates that the new architecture maintains strong robustness across various datasets and outperforms the current state-of-the-art models.

Keywords

• Artificial intelligence
• deep reinforcement learning
• DDPG
• longitudinal velocity control
• SSL