APF-DPPO: An Automatic Driving Policy Learning Method Based on the Artificial Potential Field Method to Optimize the Reward Function

Abstract

To address the difficulty of obtaining the optimal driving strategy under the condition of a complex environment and changeable tasks of vehicle autonomous driving, this paper proposes an end-to-end autonomous driving strategy learning method based on deep reinforcement learning. The ideas of target attraction and obstacle rejection of the artificial potential field method are introduced into the distributed proximal policy optimization algorithm, and the APF-DPPO learning model is established. To solve the range repulsion problem of the artificial potential field method, which affects the optimal driving strategy, this paper proposes a directional penalty function method that combines collision penalty and yaw penalty to convert the range penalty of obstacles into a single directional penalty, and establishes the vehicle motion collision model. Finally, the APF-DPPO learning model is selected to train the driving strategy for the virtual vehicle, and the transfer learning method is selected to verify the comparison experiment. The simulation results show that the completion rate of the virtual vehicle in the obstacle environment that generates penalty feedback is as high as 96.3%, which is 3.8% higher than the completion rate in the environment that does not generate penalty feedback. Under different reward functions, the method in this paper obtains the highest cumulative reward value within 500 s, which improves 69 points compared with the reward function method based on the artificial potential field method, and has higher adaptability and robustness in different environments. The experimental results show that this method can effectively improve the efficiency of autonomous driving strategy learning and control the virtual vehicle for autonomous driving behavior decisions, and provide reliable theoretical and technical support for real vehicles in autonomous driving decision-making.

Keywords

• deep reinforcement learning
• proximal policy optimization
• autonomous driving
• driving strategy
• artificial potential field method
• reward function